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ABSTRACT 

PRIOR LEARNING ASSESSMENT: FACULTY EVALUATOR TRAINING 

Abbott, Lenice C 



Methods: 



Purpose: ^fhe purpose of this study is to identify past, existing and future 
training needs specific to faculty evaluators of prior learning 
experiences. It attempts to validate the proposition that a more 
comprehensive program of continuing professional development will 
facilitate the competent assessment of experiential learning. 

A multi-pronged comparative approach is utilized to explore existing 
problems and possible solutions. Diverse perspectives on peninent 
assessment issues are garnered from institutional and extra- 
institutional research on several different levels, including assessment 
program directors and faculty evaluators. Additional input derives 
from a consulting source and an expert evaluation of the assessment 
program at National-Louis University. 

Quality assurance in prior learning assessment is contingent upon 
continuing professional development opponunities more than on any 
other single element Discrepancies exist among program directors 
and evaluators in perceived importance of philosophical orientation 
and areas of needed skill development. Increased attention to and 
discussion of these matters is critical to program viability. 

Conclusions : Evaluator participation in training and development activities must 
be encouraged and increased by recognizing and overcoming barriers 
to participation. Expanded opponunities and suppon should be 
provided by means of both formal and informal offerings. Sufficient 
priority, personnel and resources are needed to maximize 
professional expertise and assure quality in prior learning 
assessment. 



Results: 
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CHAPTER ONE 
PROBLEM, PURPOSE, PLAN, AND CONSTRAINTS 

Problem 

Background 

National-Louis University (NLU) is a private co-educational institution 
founded in 1886 for the purpose of training early childhood educators by means of 
an emphasis on classroom experiences and practical application of theory. 
Through the years, NLU (formerly National College of Education, NCE) has 
continued this philosophy of blending experience and theory in a liberal arts 
education and expanded its offerings to include not only teacher education, but 
liberal arts and sciences, allied health and human services, business, management 
and adult education on the graduate as well as the baccalaureate level. 'Today, 
National offers 13 degrees and certificates, including doctoral degrees, across 23 
departments and more than 60 academic programs" (FEP Student Services, 1989, 
p. 2). An extensive academic support system has been developed to uphold 
academic standards, increase student persistence, provide educational services and 
facilitate institutional diversity ( NCE Academic Catalog . 1986). NLU is accredited 
by the North Central Association of Colleges and Schools and in June, 1990, 
completed its transition to university status. 

The university currently serves 14,000 students annually at its campuses in 
Evanston. Chicago, and Lombard, IL; Milwaukee/Beloit, WI; St. Louis, MO; 
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McLean, VA (Washington, DC); Tampa, FL; Atlanta, GA; and Heidelberg, 
Germany (see Appendix A). Thus NLU draws its students from diverse 
geographical locations as well as educational backgrounds in response to evolving 
societal needs and changes in contemporary higher education. 

A movement toward nontraditionai delivery systems in postsecondary 
education was predicted in the 1962 Johnstone-Rivera study (cited in Miller, 
1975): 

The most important conclusion to be derived from this study is that 
America is likely to experience an adult education explosion during 

the next few decades Just as in the fifties and sixties the regular 

school system had to tool up rapidly to accommodate the greatly 
increased numbers of young persons in the population, so too in the 
seventies and eighties adult education will be subject to greatly 
increased demands ... (p. 4). 

This would seem to be corroborated by the observation of Nielsen, Assistant to 

the American Federation of Teachers President for Higher Education (cited in 

Hovey, 1988), noting "a new constituency of full-time working adults who need and 

desperately want a degree in higher education" (p. 9). 

NLU responded to that unique need in 1978 with the introduction of the 

field expenence delivery model or Field Experience Program (FEP), providing, as 

advocated later by Simosko (1985), "increased flexibility, opportunity and access to 

higher education for adults" (p. 15) by allowing students to take full course loads 

at more than 50 convenient extension sites while maintaining full-time jobs. The 

philosophy of the FEP model couples theory and knowledge with experience, 

reflecting Dewey*s view of "the soundness of the principle that education in order 
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to accomplish its ends both for the individual learner and for society must be 
based upon experience - which is always the actual life-experience of some 
individual" (1938, p. 113). Appropriate scheduling, suitable class locations, 
integrated curriculum design and cooperative student and staff support effectively 
remove barriers that frequently prevent adult students from returning to school. 
As part of its innovative approach to adult learning, the university encourages 
students to utilize its wide range of academic and support services. 

The University Library, for example, was established to provide materials, 
instruction, and services to support the institutional and research needs of students 
and faculty. Field experience students have access to NLU's own library holdings, 
including the complete Education Resources Information Center (ERIC) on 
microfiche, as well as a library network which allows them the use of many off- 
campus collections and services. Computer searches of over 300 databases are 
also available. 

The Registrar s Office evaluates transcripts and notifies students of credits 
needed for degree completion and General Studies deficiencies. Credit is also 
accepted on transcript for students who have achieved acceptable scores on 
certain standardized proficiency examinations, such as ACT-PEP, CLEP and 
DANTES, as well as for professional and military course work evaluated and 
recommended for credit through the Program on Noncollegiate Sponsored 
Instruction (PONSI) and the American Council on Education (ACE). Throughout 
the students' course work at NLU, the Registrar s Office monitors progress, 
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provides academic advising and grade reports, and coordinates graduation and 
diploma requisites. 

The purpose of the Student Services Office is to support and enrich the 
quality of education, career success and balanced lifestyle of adult learners. 
Personnel, including an alumni support team, offer information encompassing 
career, class, personal and professional activities. Career Advancement provides 
assistance in finding new positions, making career changes and developing 
professional competencies. In addition, the office publishes The Field Experience 
Newsletter and sponsors dinners and receptions, career sessions, networking open 
houses, academic and career assistance and other means of program enrichment. 

In addition to faculty, whose schedules are flexible according to student 
needs, other departments geared toward serving adult learners at NLU through 
expanded office hours include FEP Admissions, Financial Aid and Student 
Accounts. FEP Acquisitions and Distribution procures appropriate physical 
facilities for the area classes and distributes all course materials and books directly 
to students at class locations. Computer labs furnish not only access to computers, 
but a variety of software and diskettes; computer assistance is also available. The 
Center for Academic Development provides individual assistance to strengthen 
writing, reading, math, research and study skills. 

Smith (1986) points out that an additional requirement of an "adult-friendly 
college" is an assessment program to document and credit personal learning 
occurring outside a traditional educational setti ig from life experiences, to include 
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not only measurable skills and abilities, but values, attitudes and behaviors 

developed through reflection and introspection. This line of thinking is grounded, 

perhaps, in the works of adult educators such as Tough, whose interest in 

"independent learning projects" promoted the realization that nontraditional 

learning activities can be significant learning activities, or Kolb, who emphasized 

learning in real-life situations and described the steps through which adults 

proceed in experiential learning. The American Council on Education (ACE) and 

the Council on Postsecondary Accreditation (COPA) (1986) recognize that 

American society abounds in resources for learning at the 
postsecondary level .... Associations, business, government, 
industry, and unions sponsor formal instruction. In addition, 
independent study and reading, work experiences, the mass media, 
and social interaction contribute to learning and competency .... 
Postsecondary education institutions legally authorized and 
accredited to award degrees and other educational credentials have 
a special responsibility to assess extra-institutional learning as part of 
their credentialing function (p. 2). 

The Assessment Center, founded in 1978 as an integral part of the field 
experience program, coordinates the assessment and validation of non-collegiate 
postsecondary learning. It serves not only the Bachelor of Arts in Management 
program, but also those in Applied Behavioral Sciences, Allied Health, both on- 
campus and FEP, and Human Services; hence, the Center has become one of the 
university-wide activities of the Registrar's Office. .Assessment counselors and 
faculty members advise and assist students in preparing a portfolio of prior 
learning as a component of their required course work (see Appendices B and C), 
to include a resume, an autobiography, college transcripts, and descriptions and 
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documentation supporting prior learning gained outride of the traditional college 
setting. Students may incorporate learning gained through technical schools, 
seminars, workshops and training programs, as well as life and work experiences. 
Learning outcomes are assessed, and a maximum of 60 quaner hours of credit 
may be granted toward the undergraduate degree by faculty members and other 
subject-matter experts in accordance with faculty-established policies which appear 
in Appendix D. These policies include guidelines issued by ACE and the Council 
for Adult and Experiential Learning (CAEL), a pioneer in experiential learning 
research and improvement of prior learning assessment (PLA) practices. 

Current Status 

During the 1988-89 academic year, the Assessment Center processed 739 
student ponfolios. From those, 3,051 life learning experience essays were 
transmitted to faculty evaluators for credit assessment. A pool of approximately 
40 to 50 evaluators read essays in the various content areas as illustrated in 
Figure 1. 
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Fi^re 1. Essay content areas - 1988-89. 



Content Area 


Essavs Read 


Allied Health 


19 


Art 


30 


Business 


119 


Communication Arts 


40 


Data Processing 


33 


Education 


52 


Engbsh 


87 


Health Sciences 


262 


Human Services 


114 


Interdisciplinarv 


U / 


Language 


5 


iviain 


2 


Music 


18 


Philosophy/Religion 


97 


Physical Education 


98 


Psychology 


748 


Science 


39 


Social Science 


1,221 


Total 


3,051 



Figure 2 shows that the number of portfolios submitted for assessment has 
more than doubled in the past five years. The ponfolios processed duriniz 1989 
generated some 12.000 hours of credit (see Figure 3). 
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Figure 2 . Assessment Center operations. 
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Figure 3 . Assessment Center operations. 
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1988 
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The learning experience evaiuators are selected through a process 
comparable to the Council on Academic Standards' procedures for selection of 
part-time faculty in the College of Arts and Sciences; candidates without full-time 
faculty appointments are referred by departments and other evaiuators or 
identified by the Assessment Center. They complete Evaluator Information 
Forms (Appendix E), which are presented to the Assessment Center Advisory 
Committee and the appropriate division deans for review and approval. Full-time 
faculty members with appropriate expertise are added to the evaluator pool as 
they exp-ess an interest in participating. All evaiuators must hold at least a 
master's degree in a pertinent field. 

Evaiuators receive a handbook, consisting of an introduction to the Prior 
Learning Assessment (PLA) process, essay requirements, credit assessment 
options, criteria for evaluation and feedback, a sample life learning experience 
essay and related information. Evaiuators are encouraged to contact the 
Assessment Center with any questions or concerns. Occasional non-mandatory 
informational meetings have been held in the past, but are not regularly 
scheduled. 

Importance and Urgency 

While content area expertise is an unequivocally crucial requirement for 
faculty evaiuators, an understanding of and agreement with the philosophy and 
process of PLA is also imperative. Although quizzes, reports, exams and other 
formal devices, and even more experientially grounded modes such as cooperative 
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education, internships and student teaching, are familiar and comfortable means of 
assessing learning, faculty members are not generally experienced in less formal 
techniques used in prior learning assessment. They must be helped to bridge the 
gap between traditional and nontraditional while maintaining quality, validity and 
reliability. Inherent in such a necessarily flexible, highly subjective process as PLA 
are a number of potential problems, including lack of dedicated involvement, 
casual or inconsistent evaluation procedures, slow processing time, confusion about 
results, and insufficient feedback to students. 

One of the aforementioned assessment guidelines encouraged by GAEL 
states that qualifications for evaluators should be established, "all personnel 
involved in the assessment of learning should receive adequate training for the 
functions they perform, and there should be provisions for their continued 
professional development" (Whitaker, 1989, p. 79). The NLU Council on 
Academic Standards has recommended, as an enhancement to the portfolio 
assessment process, that "periodic group meetings should be held for additional 
training of evaluators and/or discussion by evaluators" (C. Fabrizio. personal 
communication, October 2. 1985). As the university moves to the forefront in 
prior learning assessment with national programs, the Assessment Center strives to 
continually increase its pool of qualified evaluators and to study, update and 
modify the existing PLA program, including the areas of evaluator selection and 
training, in accord with GAEL recommendations. NLU's scheduled 1991 
accreditation review by the Nonh Central Association of Colleges and Schools 
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lends priority to the issue and impels numerous far-reaching modifications 
affecting both the Assessment Center and the institution at large. 

Purpose 

Delineation of the Study 

The portion of the problem to be studied in this thesis encompasses a 
training and development needs assessment in an attempt to validate the 
proposition that a more comprehensive program in this area will be beneficial to 
NLU. The purpose of the study is to identify past, existing and future training and 
development needs specific to faculty evaluators of prior learning experiences. 
Needs will be studied for both inexperienced (training) and experienced 
(development) evaluators. The study will emphasize primarily what Laird (1985) 
refers to as macro training needs, that is, those which exist in a large group of 
personnel or an entire population; micro, or individual, training needs may also 
be generated from the research. 

In order to meet this objective, research will be conducted at NLU and 
other institutions offering nontraditional prior learning assessment. Whitaker 
(1989) contends that appropriate standards of professional practice must extend to 
all those involved in the expen judgment of experiential learning. Thus, faculty 
consensus on policy and peer review of evaluator practices are essential elements 
of any assessment program. Funherraore, an appropriate institutional body or 
official, namely an assessment staff and/or director, should accept clear 
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responsibility for quality assurance and monitoring of procedures. Hence, needs 
will be examined from the perspectives of (a) evaluators and (b) assessment staff. 
Various methods of meeting these needs will be identified. The results should 
provide insight into perceived and real evaluator needs, as well as direction for 
intensifying and invigorating NLU's existing training procedures. 

Conceptual and Operational 
Definitions of Variahlp<: 

Prior learning assessment fPT A) This is conceptually defined as the 
process by which learning occurring outside a college classroom from professional 
or personal experiences is identified, evaluated and equated with college credit 
(Knapp and Jacobs, 1981). It is operanonally defined by institutional guidelines in 
place for detailing, describing, evidencing and documenting unsponsored learning 
through a ponfolio or similar narrative instrument. 

is conceptually defined as the expert who makes a 
professional judgment regarding the acquisition of prior learning and recommends 
appropriate academic credit. Operanonally it is defined by the various groups 
involved in PLA at panicipaung insfitutions and is idenfified as assessment staff 
member, faculty member, faculty committee or outside expert. 

Training and development needs . This is conceptually defined as those 
areas of deficiency in skills and abilities which can be remediated through 
instruction in order to increase professional expertise. It is operationally defines 
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as the areas identified by assessment staff and evaluators which need to be 
addressed in order to facilitate the valid and reliable assessment of prior learning. 

Training and develop ment activities. This is conceptually defined as 
individual or group pursuits designed to maintain and improve job performance. 
It is operationally defined by the ways in which evaluators learn about the PLA 
process, both initially and on a continuing basis. 

Quality assurance. It is conceptually defined as the process of ensuring that 
goods and services meet predetermined standards (Stoner and Wankel, 1986). It 
is operationally defined as the responses indicating provisions for reviewing and 
appealing credit recommendations in the PLA process in order to maintain 
validity and rehability. It is funher defined by such specifics of processing as 
feedback to students and evaluation turnaround time. 

Research Questions 

1. Who comprises the faculty involved in the evaluation at prior 

learnmg experiences, and what are their relevant credentials? 
1 How and by whom should evaluator training and development needs 

most appropriately be ascertained? 
3. Which specific training objectives need to be addressed in 

conjunction with evaluator responsibilities, as perceived by: 

a. assessment administration? 

b. evaluators themselves? 
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4. What functions do provision of adequate training and other 
compounding factors discharge in the performance of expert 
evaluators? 

5. How and to what extent is quality assurance in the assessment 
process impacted and maintained through evaluator training and 
development? 

6. What are the prevailing attitudes among faculty evaluators and 
assessment administration regarding evaluator training and 
development? 

A pplication of Resnlt-s 

The results of this study will be submitted to the Assessment Center 
Director as partial fulfillment of a charge to assist in staff training and faculty 
development. Specific strategic recommendations will be made regarding 
evaluator training and development at NLU. Ultimately, in cooperation with the 
Director, a program to augment evaluator training will most likely be developed 
and implemented; the first session is tentatively scheduled for November, 1990. 
This ongoing process will enhance the current program and facilitate the 
competent assessment of experiential learning. 

Plan 

Research Design 

This study utilizes a multi-prnnged, comparative approach to explore 
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existing problems and solutions. Research will be conducted both institutionally 
and extra-institutionally on several different levels which will furnish diverse 
perspectives on peninent issues and provide varied input from which to draw 
conclusions and make recommendations. Figure 4 clarifies the various levels of 
the research design. 
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Population Description 

Within NLU, the population consists of several sectors involved in the PLA 
process: 1) the Assessment Center Director, 2) ail active faculty evaluators 
(approximately 40) and 3) the Director of the Center for Educational Systems and 
Services, which provides consulting expertise in nontraditional programs for adult 
learners to other educational institutions, 

Extra-institutionally, the population is comprised of assessment directors: 
1) from schools which are contracting with NLU s Center for Educational Systems 
and Services for degree completion programs (displayed in Appendix A, 
Educational Contracts) and 2) from other institutions with PLA programs. In all 
cases, evaluations must be accomplished through a narrative format similar to that 
in use at NLU. Six contract school assessment directors and eight other directors 
will be contacted. Following is a list of targeted participants ("C* indicates 
contract school): 

1. St. Edward's University - Joseph O'Neal 

2. Empire State College - Martin Thorsland 

3. Thomas A Edison State College - Deborah Dagavarian 

4. City Colleges of Chicago - Sylvia Bush 

5. College of St. Francis - Janine Hicks 

6. Trinity College - Diane Gingrich 

7. Governors State University - Otis Lawrence 

8. Metropolitan State University - Leah Harvey 
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c 


9. 


Friends University - Roben Ordway 


c 


10. 


Calumet College - Dan Mack-Ward 


c 


11. 


Concordia Collepe - Jeanette (^Innlcpv 


c 


11 


Geneva College - Andrea Korean 


c 


13. 


Nyack College - Kathy Vandenbergh 


c 


14. 


Wheeling Jesuit College - Carolyn Dalzell 



All expert evaluation of NLU's program will also be examined. This 
independent assessment of overall needs was recently completed by Urban 
Whitaker, Founding Member of the Council for Adult and Experiential Learning 
and GAEL Regional Executive Officen and will be summarized insofar as its 
applicability to this study. 

Instrumentation 

Cover letter. A cover letter will be sent to each subject as an introduction 
to the questionnaire, with different formats for assessment directors and faculty 
evaiuators. Some of the directors^ letters will include personalized notes based 
upon networking at professional meetings and institutional contract relationships. 
All cover letters will explain the purpose of the study as well as how the results 
will be used, and will be appropriately modified Appendix F exhibits a sample 
letter to a director who has been personally contacted and has agreed to 
participate in the study. In Appendix G, an evaluator letter offers the possibility 
of future training sessions as an incentive to compliance. 
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Evaluator training survey for assessment directors . The Evaluator Training 
Survey appears in Appendix H. It defines the term evaluator (assessor) and seeks 
to identify specific practices and procedures in use. The items are arranged, for 
the most part, in sequence with the Research Questions previously set fonh. 
Demographic items are included to proffer more detailed descriptions of subject 
institutions' PLA programs. Several questions have been adapted from 
Procedures for Revie wing an Assessment of Prior Learning Progrnm , a publication 
of the Wisconsin Extended Degree Comprehensive Assessment Plan (EDCAP). 
Appropriate modifications have been made in the faculty evaluator version, shown 
in Appendix I. This questionnaire also follows in sequence with the Research 
Questions, but revises items for evaluators' personal responses. A few items have 
been amended or deleted as deemed necessary for applicability to this population. 
The survey was pilot-tested by an evaluator to check wording and format and to 
arrive at an approximate completion time, of which respondents are informed in 
the cover letters. So as to minimize confusion upon return, the director survey 
will be printed on yellow paper and the evaluator survey on green. 

Data Collection Procedure 

The list of participants has been finalized in consultation with the Directors 
of Assessment and Educational Systems and Services, and the cover letters and 
questionnaires will be mailed to assessment directors and faculty evaluators via 
U.S. or intercampus mail. If completed surveys are not received within one week 
of the requested return date of August 20, 1990, non-respondents will be 
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contacted by telephone and encouraged to reply. A response rate of 90 to 100 
percent is anticipated among directors, but the evaluators' response rate will be 
ostensibly lower. 

The Director of Educational Systems and Services may be asked to 
complete a director questionnaire, but a personal interview based loosely on the 
survey may prove more germane. The expert evaluation will be summarized from 
a study completed at NCE in December, 1989, with further clarifications solicited 
if needed. 

Data Analysis Techniques 

Data will be transformed to numeric coding and entered into an IBM-PC 
for processing by the Statistical Package for Social Sciences (SPSS). Category 
data will be derived from the variables involved, which are measured on either 
nominal or ordinal scales: therefore, nonparametric statistical techniques will be 
used. Descriptive statistics will be employed to describe the population and 
summarize its training needs. Numbers, modes, percentages and categories v/ill be 
examined and distributions illustrated by means of such displays as frequency 
distributions, pie charts, cross-tabulations and block diagrams. 

Schedule 

The Gantt Chart of Events appears in Appendix J. showing that the 
research study will be completed so as to meet the deadlines for December 
graduation. 
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Budget 

The Line Item Budget is shown in Appendix K- It indicates that, while the 
bulk of the research expenses per se will be covered by ihe Assessment Center, 
word processing fees will be paid from the researcher's personal funds. 

Organizational Support 

A copy of the letter of support from NLU's Assessment Center Director is 
shown in Appendix L. It validates the high priority of the thesis topic, offers 
assistance in conducting the study and specifies available resources. 

Dissemination of Finding s 

The completed thesis will be submitted to the Assessment Center Director 
in discussion of and planning for faculty evaluator training sessions. It should 
provide both a representative overview of PLA programs in place and a more 
detailed picture of needs specific to NLU. Findings and recommendations will be 
shared with participants if requested. 

External outlets for dissemination of the thesis findings will also be sought 
in order to contribute to the body of knowledge in the field of the assessment of 
prior learning. CAEL plans to conduct a national survey which might incorporate 
elements of this study. Its Annual International Assembly in November, 1990, 
might be another appropriate forum. The possibility of presenting papers at 
CAEL or other professional meetings will be explored. 
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Limitations 

The attempt to quantify evaluator training needs limits, to an extent, the 
breadth of possible responses. Provisions have been made to elicit additional 
ideas where feasible, 

A comparative lack of experience in PLA on the part of a portion of the 
contract school directors will probably present another limitation to the study. 
Although this segment of the population has expressed concern with this issue, few 
have addressed their concerns in the relatively short duration, to date, of their 
programs (CESS Annual Degree Completion Program Conference, April, 1990), 
Most of the faculty evaluator development has been conducted by NLU's Center 
for Educational Systems and Services, producing potentially similar responses. 

Delimitations 

The decision to sample only 14 outside institutions, due primarily to time 
constraints, might limit generalizability to other institutions" PLA programs that 
could be produced in a larger-scale study. It will not, however, hamper a needs 
assessment for NLU's own program. 

The restriction of the research to assessment directors and evaluators, 
excluding other assessment staff members (i.e., counselors), eliminates a 
potentially useful source of feedback. Assessment counselors typically process and 
expedite assessments, facilitate contact between students and evaluators, answer 
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questions and interpret results and recommendations for students. In these 
capacities, counselors work closely with evaluatcrs and observe the inconsistencies, 
confusion, indecisiveness and other difficulties intrinsic to the PLA experience. 
Although counselors would be able to suggest problem areas and potential 
training solutions in response to the quality assurance issue, for this research, their 
involvement would not render substantively different information. Their 
contributions might be useful in future research; NLU assessment counselors will 
presumably panicipate in upcoming faculty evaluator training and development 
sessions. 
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CHAPTER TWO 

PROBLEM EXPLICATION, THE THEORY AND ITS EMPIRICAL 
POTENTIAL, AND NEED FOR STUDY 

Problem Explication 

During the past 20 years, colleges and universities have, in an effort to 
better meet the needs of increasing numbers of adult learners, become active in 
the assessment of personal learning occurring through life experiences. The issues 
involved in crediting non-collegiate postsecondary learning are numerous in terms 
of assuring academic viability and integrity. It has become incumbent upon 
providers of such nontraditional alternatives to maintain cadres of trained faculty 
evaluators who are not only skilled in the execution of prior learning assessment, 
but more fundamentally, who maintain a tacit understanding of the ramifications 
of experiential learning in higher education today. 

Organizational Context 

National-Louis University provides prior learning assessment services yearly 
for hundreds of adult students through the Assessment Center, located at its 
Lombard campus. The Center is staffed by a Director, four Assessment 
Counselors, an Administrative Assistant and an Office Assistant and coordinates 
the endeavors of approximately 40 active faculty evaluators. This study focuses 
upon that team of evaluators and their past, present and future training needs 
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relative to assessing prior learning. For the sake of comparison, this researcher 
has funher chosen to study evaluator training needs at 14 other institutions of 
higher education. Consultative recommendations will also be studied to clarify the 
role of evaluators in the experiential learning configuration. 

History of the Problem 

Experient ial learning . The distinction between formalized instruction and 
learning derived from personal experience is hardly a new one. The historical and 
philosophical roots of experiential learning are strong. From earliest Egypt and 
Babylon, for example, experiential learning has remained crucial to training 
programs in various crafts and trades. Education in ancient Greece involved, 
rather than strictly transmission of knowledge, encouragement of participation in 
and observation of life and, as espoused by Plato, " only by helping ... put in 
proper form the tacts that . . . experience had already given" (Houle, 1976, p. 20). 
The Chinese, in the second century B.C., developed one of the earliest formalized 
systems of learnmg assessment, which endured for over two thousand years and 
tested practical qualities for leadership in the national public service. "By the 
period of the Sung Dynasty from 960 to 1279 A.D., when European universities 
were slowly coalescing into their original primitive structures, the assessment 
system of China was already highly sophisticated" (Houle, 1976, p. 20). 

"The patterns of modern education were, to a remarkable degree, either 
born in or nunured by the Middle Ages" (Scherich, 1973. p. 38). Houle (1976) 
identifies several systems of advanced learning which accepted personal 
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experiential learning and developed during medieval times in Europe. 
Undoubtedly the apprenticeship system of craft guilds, based entirely upon 
pragmatic experiential learning, was one of the most imponant and enduring. 
Under this system, learners worked for master practitioners, acquiring knowledge, 
competence and appreciation of excellence within their crafts. The guilds 
established the training programs and regulated the nature and sequence of 
instruction. Learners progressed through the ranks of apprentice, journeyman and 
master by demonstration of knowledge and skill. The guild system of 
apprenticeship perseveres in Europe today and serves as a model for modern 
American trade union and vocational/technical training programs which supervise 
learning, certify quality and regulate conditions of employment. 

The chivalric system, too, was almost entirely experiential. The general 
pattern of education evolved between the ages of 7 and 20, as learners progressed 
from personal servants to squires, ultimately achieving the status of knight. New 
skills were mastered at each level in the elements of courtly life, religious devotion 
and fighting. Chivalry imparted the knowledge base imponant for noblemen 
through "hands on" training followed by competency based assessment. 

The final mode of medieval experiential education identified by Houle "was 
a sporadic and often unorganized continuation of learning throughout life that 
occurred in monasteries, courts, and private libraries" (1976. p. 24). Aduh 
scholars such as Machiavelli sometimes sought to supplement their studies by 
means of the distillation from direct experience of underlying concepts and 
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principles. Although this form of education never developed into a single system 
per se, it was, perhaps, the precursor of modem self-directed study. 

In subsequent centuries education, including experiential learning, has 
undergone constant modification. Societal forces such as nationalization, 
industrialization and the rise of relatively widespread literacy manipulated its 
nature, scope and form. New educational patterns and institutions evolved as a 
consequence of societal growth and transformation. University tradition began to 
give way in light of the demise or alteration of other educational systems based on 
experiential learning. Self-guided lifelong learning became inadequate, and by the 
second half of the nineteenth century an urgency had developed in colleges and 
universities both overseas and in the U.S. for a combination of systematic scholarly 
instruction and experiential learning. John Stuart Mill, in his 1867 inaugural 
address as Rector of St. Andrews University in Scotland, eloquently redefined 
education, distinguishing between formal instruction and experiential learning. 
"Whatever helps to shape the human being - to make the individual what he is, or 
hinder him from being what he is not - is part of his education" (cited in Houle, 
1976, p. 27), 

Through the remainder of the nineteenth century and into the twentieth, 
debate continued over the merit of integrating theoretical with practical study. 
Houle (1976) cites the transcript from a conference of college officials in Chicago 
in 1871, which "shows that some of the excesses of experiential learning with which 
we are all too familiar today were also practiced by our professional ancestors" 
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(p. 28). Then, as now, the relevance of experience to the conceptual was 
scrutinized, yet planned experiential leamv.g flourished. 

Supplementary firsthand experience and laboratory experimentation began 
to emphasize practical application of knowledge, A pioneer in this practice was 
William Osier, a Johns Hopkins University Professor of Medicine who required his 
students to perform autopsies and observe patient treatment in the hospital wards. 
Internships, practica, guided simulations and field work became essential elements 
of training in many professions. Moreover, technical and vocational schools 
proliferated to meet the demands for assimilation of specialized skills and 
theoretical understanding. 

The mid-twentieth century saw dramatic changes in postsecondary 
education, precipitated by changes in the society and in the educational objectives 
of students. "Community colleges and vocational institutions multiplied in number 
and became vital elements of the system, broadening access to learning and 
providing diverse educational opponunities. Interest in nontraditional education 
encouraged more flexible programming. Learning in extrainstitutional 
circumstances became more common" (Miller & Mills, 1978, p. 4). 

Houle identifies Worid War II as the genesis of the trend "to educate the 
student so that he knows not only how to solve problems but also how to deal 
with normal conditions of life" (1976, p. 30). Thus field trips, clinical training, lab 
instruction, ans series, student teaching, conducted tours, study abroad and other 
nonclassroom activities have become important avenues for combining abstract 
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and concrete learning through sponsored educational experiences. As the 
American education system expanded to incorporate these elements into the 
curriculum and integrate them into the credit system, new frameworks and 
methodologies emerged, resulting in one modem system embracing both 
theoretical and experiential learning. 

Alternative programming . This expanded modem system, however, still 
fails to completely meet the needs of certain kinds of students, most of whom are 
adults. Houle (1976) remarks on four such groups: 

1) those who acquired their experience before their theory 

2) those who prefer or require self-directed, self-guided learning 

3) those who desire guided but personalized, and usually experiential, 
study programs and 

4) those who wish to compile a record of previous learning through 
various means into the basis for further study toward a college 
degree. 

Even in the late 1800s, a few postsecondary institutions recognized this 
failing and attempted to meet the needs of adult students who found the 
traditional college or university model lacking through opportunities for 
independent study or waiver of subject area requirements. This type of situation 
intensified significantly after Worid War I, when extension degrees became 
increasingly common, and again after World War II, when systems such as the 
Advanced Placement Program (APP) and the College Level Examination Program 
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(CLEP) were instituted to award college credit on the basis of educational 
assessment of prior experience. 

CLEP, administered by Educational Testing Service, is probably the best 
known credit-by-exam program. Others include the ACT - Proficiency 
Examination Program (ACT-PEP), administered by the American College Testing 
Company, and Defense Activity for Non-Traditional Educational Support 
(DANTES), originally available only to military personnel but now available to 
civilians as well. Faculty members from a variety of institutions determine subject 
matter, write the tests, which are similar to final exams in undergraduate courses, 
administer them to develop national norms and assist in establishing academic 
policy in regard to these interinstitutional examination programs. In addition, 
colleges such as Ohio University, Thomas A. Edison State College and the 
University of the State of New York (New York Regents) sponsor other large 
scale credit-by-^xam opportunities. All of these standardized achievement tests 
are designed to reflect college-equivalent course work in specific subjects and are 
administered nationally at college and university testing centers. 

In 1954, Brooklyn College was perhaps the first undergraduate 

postsecondary institution to award credit to adults based upon faculty assessment 

of previous experience. The University of Oklahoma in 1957, and others soon to 

follow, began to design degree programs specifically for adults based upon 

a shared belief in the educative power of direct and usually 
unguided experience, a value that Mill celebrated more than a 
century ago and that Adlai Stevenson expressed even more strongly: 
"What a man knows at sixty that he did not know at twenty may boil 
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down to something like this: The knowledge he has acquired with 
age is not the knowledge of formulas, or forms of words, bur of 
people, places, and actions • a knowledge not gained by words, but 
by touch, sight, sound, victories, failures, sleeplessness, devotion, love 
- the human experiences and emotions of this earth and of oneself 
and of other men; and perhaps, too, a little faith, and a little 
reverence of things you cannot see'' (Houle, 1976, p, 32-33). 

The evolving experiential learning movement continued to gain momentum, 

becoming quite widespread during the 1970s. In fact, the burgeoning practice of 

"awarding academic credit based on noncollege learning achievements signals one 

of the major learner-centered innovations of the 1970s'' (Cargo, 1982, p. 1). 

Mandell and Michelson (1990) also see this practice as one of the most significant 

recent breakthroughs in higher education. 

Acknowledging the educational validity of students' prior learning 
and translating that learning into college credit has both righted the 
traditional injustice against individuals whose education has taken 
place outside the academy and enriched the academy itself. In 
institutionalizing prior learning assessment, colleges and universities 
have given new life to the interaction among cultures of knowledge 
and informed our understanding of the relationships between 
learning and practical activity (Mandell & Michelson, 1990, p. viii). 

Harking back to John Dewey (1938), these innovations recognized that people are 

shaped and modified by their experiences and have the ability to learn from them. 

Furthermore, they acknowledged that "an increasing amount of lifelong learning is 

taking place outside the traditional academic environment." (Dychtwald, 1989, 

p. 150), and that in response to the diversity in student population, "the form and 

function of education will need to conform to the more adult lifestyle" (Dychtwald, 

1989, p. 151). 

External degree programs began to be cultivated in the 1970s as rejoinders 



Er|c " 43 



33 

to the drastic changes in student cHentele. These programs have embraced radical 
alterations in curricula and institutional arrangements by allowing students to 
complete degree requirements "by virtually every possible means of earning credit, 
in accordance with the guidances of the institution'* (Cabell & Hickerson, 1988, 
p. 140). 

Empire State College of the State University of New York, for example, 
was the first major college designed specifically for adults, utilizing both 
individualized instruction and in fact, individualized degree programs to grant 
credit for college-level learning irregardless of how it was gained. Learning 
contracts are widely employed to measure experiential learning outcomes at this 
alternative institution founded in 1971. 

In the same year. Metropolitan State University in St. Paul, MN, was 
organized as an upper-divisional university with the mission to serve adult 
students. This is accomplished through the framework of flexible, competency- 
based education and assessment. One of the offerings of DePaul University in 
Chicago is a similar competence-based degree for adult learners. 

The University without Walls was established at the University of 
.Massachusetts (Amherst) in 1972 to open the university to those who were eager 
to learn but had previously been excluded due to discrimination, logistics, or other 
reasons. This external degree program offers portfolio development and prior 
learning assessment as integral components of individualized college degrees. 

Unconventional institutions such as these have been developed to extend 
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new types of learning experiences to new clienteles in unaccustomed ways. All of 
rhese programs are organized on an achievement basis, and many specify no 
definite time for completion. Components include flexible scheduling, credit for 
prior learning, credit by exam, contract learning, and individualized degree 
planning. More difficult to fit into the conventional credit conception, "these 
challenges put a premium on innovation and flexibility and force the 
administrative process to be adaptable to new ideas" (Folger, 1978, p. 204). 

The early classical curriculum of the university "was rigidly prescribed as to 
sequence, content, and location with few options available to the student" (Knapp, 
1977, p. 1). In the 1970s, though, universities continued the crusade begun more 
than a hundred years earlier "to try to deal more creatively than before with 
experiential and scholarly learning, balancing them in various curricula and 
institutions" (Houle, 1976, p. 33). Initially that meant stretching beyond isolated 
textbook instruction toward direct supplementary experiences. Now it includes 
examining "how guided or unguided experience (often community-based) can 
provide the skill, knowledge, and sensitiveness that colleges and universities can 
accept as wonhy accomplishments, even, in some cases, for the full satisfaction of 
degree requirements" (Houle, 1976, p. 33). 

Houle (1976) suggests that this shift in philosophy once again entails the 
invention of many innovative structures, concepts and processes and most 
importantly, cultivation of a new, accepting attitude that appreciates the 
educational value and worth of aU experience. Smith (1986) concurs that active 
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learning does, indeed, take place outside the college and identifies these three 
characteristics of such learning: 

1) it is personal 

2) it is purposeful 

3) it is powerful 

"Personal learning continually changes (the learner), developing behavior, skills, 

knowledge and attitudes into a unique set of hidden credentials" 

(Smith, 1986, p. 20). 

Learners seldom think, however, about the importance or even the 

existence of this type of learning or its impact on their lives. Hence, colleges and 

universities are redoubling their efforts to help adult learners identity, analyze and 

evaluate personal learning and to benefit from the empowerment that comes with 

its recognition. In the 1980s more than 500 colleges were identified which 

awarded credit through prior learning assessment (Hyenga, Adams & Rowe, 1981; 

Simosko, 1985), with Cargo (1982) estimating that as many as one in three 

colleges and universities was using assessment to grant credit. Certainly these 

numbers have multiplied in successive years with the continual expansion in design 

and implementation of PLA programs. 

The faculty and administration involved have experimented with 
alternative structures . . . and, in the process, evolved a whole new 
area of academic practice. The institutions that have been involved 
run the gamut of American higher education: from small, private 
institutions to large state systems; from residential campuses to 
external degree programs; from community colleges to universities. 
In some cases, the institutions have their origins in the educational 
experimentation of the early 1970s and an explicit commitment to 
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nontraditional students. Others have only recently focused on new 
populations of students for whom the evaluation of prior learning is 
central to academic endeavor (Mendel & Michelson, 1990, 
p. 50). 

Although it is not certain precisely how many colleges currently offer older 
students credit for prior learning, Wilson (1990) estimates the number at 700 to 
1,000. "The plans started on campuses about 15 years ago and have blossomed as 
the number of nontraditional students either entering college for the first time or 
returning to finish their degrees has grown" (Wilson, 1990, p. A35). Simosko 
(1988) cites institutional and foundation support (W.K. Kellogg, Ford and 
Carnegie, among others) as significant contributors to the spread and acceptance 
of prior experiential learning. 

In conjunction with assessment of prior learning, numerous additional 
credit-bearing options have been cultivated. Departmental challenge exams and 
faculty-created tests are available at most institutions. Evaluation of various 
licenses and certifications is often possible. External degree programs 
incorporating distance learning, college correspondence courses and independent 
studies are flourishing. Validation degree programs such as Thomas A. Edison 
College of New Jersey and the New York Regents Degree continue to prosper. 
"Never before jn the history of the United Slates have there been so many flexible 
opportunities for adults wanting or needing college credit or a degree" (Simosko, 
1985, p. 16). 

The Project on Noncollegiate Sponsored Instruction (PONSI) has greatly 
furthered the cause of PLA. In 1974, this joint venture of the American Council 
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on Education (ACE) and the New York Board of Regents implemented the 
evaluation of corporate and professional training programs for college credit 
equivalency. At the request of sponsoring agencies in business and industry, ACE 
member faculty judges the quality of traditionally operated formal courses in 
noncollegiate settings. Teams of reviewers conduct evaluations, produce course 
descriptions and establish recommendations for type and amount of credit to be 
awarded; results are published annually in ACE's National Guide to Credit 
Recommendations for Noncollegiate Courses . 

ACE similarly effects a national program to establish credit equivalencies 
for military training and formal courses, although this program began as early as 
1945. Military occupational specialties (MOS) are also reviewed in accordance 
with ACE policies and credit recommended if deemed appropriate. The annual 
Guide to the Evaluation of Education Experience in the Armed Services publishes 
all recommendations. 'These (ACE) credit determination methods are firmly 
established and are used by most colleges and universities" (Cargo, 1982, p. 1). 

A more flexible, more individualized means of establishing prior learning 
that links student knowledge and skills to goals and educational programs is 
portfolio assessment. "A fundamental concept that underpins most portfolio 
assessment programs is that the assessment process itself should have significant 
educational value" (Cargo, 1982, p. 20). While the portfolio process may 
incorporate other credit determination methods such as ACE/Military courses or 
challenge exams and specific guidelines and requirements vary widely, the 
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portfolio customarily entails written documents analyzing, describing and 

documenting college-level competence. "Although students are not guaranteed to 

receive academic credit for their noncollege learning based solely on completing a 

portfolio" (Cargo, 1982, p. 20), it provides another viable alternative for the adult 

student seeking to communicate and validate prior learning. 

This, then, is the essence of prior learning assessment through alternative 

programming - to provide formal recognition for extrainstitutional learning by a 

variety of means. 

Full and effective use of all educational resources is a worthy 
educational and social goal. Social equity requires that equivalent 
learning regardless of where and how it is achieved be incorporated 
into the system of rewards for learning and competency .... 
Through reliable and valid assessment of extrainstitutional learning, 
more persons can be enabled to enter college and work successfully 
toward degrees and other academic goals (American Council on 
Education, 1987, pp. 2-3). 

Implications for prior learning assessment . Because nontraditional 
programs in PLA developed so rapidly in the 1970s, they were the focus of a great 
deal of national attention. This rapid growth and diversity of experiential 
programs coupled with the use of experiential learning assessment as a recruiting 
and marketing device have resulted in serious misgivings about credentialing 
standards (Knapp & Jacobs, 1981). As demand has increased for college degrees, 
institutions with less than adequate standards offering "quick and dirty" degrees 
have multiplied concomitantly. 'These institutions, which may be licensed to do 
business but have not been accredited or accepted as candidates for accreditation 
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by federally recognized accrediting bodies, should not be confused with institutions 
with fine programs that offer • • • credit for noncollege learning" (Cargo, 1982, 
p. 2). This tendency to confuse reputable assessment programs with "diploma 
mills" is one result of national exposure and demand. 

The Task Force on Educational Credit and Credentials undertook a two 
year study (1974-1976) to determine how postsecondary education's system for 
awarding credit and credentials can be modified or improved to best meet current 
educational and social needs. In 1977, the Task Force issued a report with two 
major purposes: 

1) to increase awareness of the need to accommodate learning attained 
in a variety of settings and under a variety of sponsorships, and 

2) to recommend changes that would improve the qualit)' of 
information conveyed by academic credentials through adoption of 
effective and consistent policies and practices. 

Several of the Task Force s fifteen recommendations address the issues of 
nongovernmental accreditation through peer review, development of alternative 
programs as valid means of certifying educational accomplishment while 
promoting lifelong learning, and, in the evaluation of extrainstitutional learning, 
"demonstration of accountability to all constituencies - students, employers, 
external monitoring agencies, government, peer institutions and others who 
support (institutions) or have an interest in their operations" (Miller & Mills, 1978, 
p. 242). 



After years of rapid growth in both adult degree programs and prior 
learning assessment, educational institutions themselves are calling tor guidelines 
and assistance in improving the quality of their programs. Adult learners, their 
sponsoring employers, and the accrediting community are also paying increasing 
attention to quality assurance. New initiatives in the areas of improved practices 
and quality assurance are being undenaken. Raising standards, however, must be 
based upon certain underlying assumptions, the first of which lays the foundation 
for assessment: 

1) Adult capability for college-level learning independently 

2) Academic integrity in practice and perception 

3) Acknowledgement of genuine extrainstitutional learning 

4) Appropriately qualified personnel for assessment 

5) Commonality among practitioners 

6) Protection against conflict of interest and unethical processes 

7) Effective evaluation and re-^jvaluation of policies and procedures 
Only through acceptance of these assumptions in their entirety can institutions 
hope to establish and maintain the highest standards in prior learning assessment. 
(Tate, 1990). 

Credentialing bodies . Accredited colleges and universities with alternative 
educational programs subscribe to national and regional guidelines in setting 
program standards and procedures, according to Cargo (1982), and students 
report a high degree of satisfaction with them. The American Council on 
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Education (ACE) is the primary coordinating body in the U.S. for postsecondary 
education, working through voluntary and cooperative actions. 

At least two ACE components concern themselves specifically with 
alternative educational programs for adults. The Commission on Higher 
Education and the Adult Learner focuses on issues of public policy, institutional 
self-assessment and cooperation among associations dealing with adult learners. 
The Commission on Educational Credit and Credentials gives continuing attention 
to improving the quality of information conveyed by these means. As an arm of 
the Office of Educational Credit, the Commission fosters high standards and 
sound practices for evaluating and acknowledging extrainstitutional learning. 
Some of the aforementioned credit equivalency programs such as PONSI and 
CLEP operate under the auspices of this Commission. Its Statement on Awarding 
Credit for Extrainstitutional Learning is a schematic for developing policies and 
procedures. 

The Council on Postsecondary Accreditation, while not an accrediting body 
itself, evaluates national and regional accrediting groups, recognizing those which 
meet criteria of quality and performance. It publishes an annual compendium of 
institutions and programs which have, in turn, been evaluated by the recognized 
accreditors and determined by their peers to meet acceptable levels of educational 
quality. 

Regional accrediting associations, which function through voluntary 
compliance, and self-regulation, emphasize educational quality through the 
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evaluation of institutional programs in relation to the program's own goals and 
objectives. These accrediting bodies, which work to formulate procedures and 
criteria that promote sound education, were initially uncomfortable with 
alternative programs (Knapp & Jacobs, 1981). However, ''the trend has been 
toward greater flexibility in the application of standards and the adoption of 
procedures for considering and evaluating innovative and nontraditional programs 
as special cases" (Miller & Mills, 1978, p. 213). Rather than setting 
comprehensive standards, regional associations require institutional standards for 
credit to be based upon sound educational principles. 

The majority of state coordinating and governing agencies for higher 
education review and approve new programs, and many also have procedures for 
reviewing ongoing programs. These procedures usually relate to factors such as 
need, cost, potential quality and relationship to other programs within the state. 
Such state chartering and licensing measures serve to legitimize operations in 
alternative education. 

These and other diversely intentioned organizations and associations, albeit 
essential to the field of higher education, do not completely satisty the need which 
has arisen with the advancement of prior learning assessment for reliability and 
credibility within this new arena of specialization. Keeton (1976) points out the 
need for accepted standards of credentialing experiential learning, recognizing that 
testing and criticism are essential to the justification of such efforts. He further 
maintains that improvements in assessment and credentialing will "enlarge our 
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vision of the aims of education" (Keeton, 1976, p. 18)* 

Due to a similar concern for practices and standards of the 1971 
Commission of Non-Traditionai Study, the Cooperative Assessment of Experiential 
Learning (CAEL) was formed in March 1974. It began as a three year research 
project of Educational Testing Service (ETS) in Princeton, NJ, and was funded by 
a grant from the Carnegie Corporation. CAEL*s objective was to test the 
feasibility of doing valid and reliable assessment of learning occurring beyond the 
college classroom, and the project was carried out by ten colleges, ETS and a 
committee of 15 educators. The needs on which they focused were threefold: to 
know what was happening in experiential learning across the country, to deduce 
how to evaluate its outcomes and to find ways to meet the demand for swift 
improvement in practices. 

During the ensuing years from 1974 to 1977, 243 postsecondary institutions 
became active members in CAEL, with 27 of them serving as research work field 
sites and others developing their own assessment of prior learning models 
(Council for Adult and Experiential Learning, 1986). A multitude of working 
papers, formal publications, student guides, faculty handbooks, accounts of 
institutional models and technical reports resulted from the project. 

At rhe end of that period of research and development, and through the 
urging of participating institutions, CAEL was chartered with the Regents of the 
State of New York, becoming, in February 1977, an independent non-profit 
association of institutions and individuals called the Council for the Advancement 



44 

of Experiential Learning. A new objective accompanied the new name: to give 
equal priority to the assessment of experiential learning and to the development of 
improved learning opportunities. 

Over the next five years more than $1,500,000 was awarded to GAEL by 
the W.K. Kellogg Foundation for institutional development of colleges and 
universities in their use of experiential learning; over 750 institutions utilized 
these funds (GAEL, 1986). Grants from the Lily Endowment and the Fund for 
the Improvement of Postsecondary Education also furthered the mission of GAEL 
In cooperation with the Kellogg Foundation and partner institutions, GAEL 
mounted a nationwide effort between 1979 and 1985 called Project LEARN, 
developed to expand both traditional GAEL services and new adult learner 
services. 

In 1985, another name change sought to more precisely reflect CAEL's 
current commitments, and the Council for Adult and Experiential Learning 
emerged as a leader in the realms of experiential learning assessment and the 
expansion and improvement of educational services for adult learners. Today 
members hail from 50 states and from five foreign countries. Beyond providing 
services and information to these members, GAEL serves additional institutions 
and educational professionals yearly through national assemblies, workshops, 
training and orientation meetings, public presentations, a newsletter and numerous 
other publications. Many different vehicles are available, through GAEL Gentral 
and its 20 regional offices, by which members can "contribute to a rapidly growing 
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national effort to provide increased educational opportunity to adult learners" 
(GAEL, 1986, p. 2). 

As the major quality assurance medium in the field of experiential learning 
assessment, GAEL has answered Gartner*s suggestion for a set of standards which, 
although not intrinsically superior or inferior, are different from the norm. 'The 
task, then, is clean to establish standards and procedures of assessing and 
credentialing experiential learning that have a quality of rigor of their own - 
separate from but equivalent to those used regarding the traditional mode of 
learning" (Gartner, 1976, p. 40). The continual updating of such standards is one 
of GAEUs principal aims* 

Gurrent Status of the Problem 

The surge in numbers of adult college students has caused many institutions 

to emphasize programs appropriate to specific educational objectives of the new 

clientele, as predicted by Ferguson (1978). His challenge endures to make reality 

the goal of integrating the reliable and valid assessment of students' achievements 

as a systematically applied component of the educational program. Among 

institutions, a good deal of diversity characterizes the development of assessment 

strategies. Inc^eed, 

one virtue of our decentralized system of education is that individual 
institutions can adopt new approaches, and administrative changes 
can be made to accommodate new bases for credits and credentials, 
new ways of managing the education process, and new roles for 
faculty members (Folger, 1978, p. 215). 

Specific procedures, educational philosophies and policies vary widely. 
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Nonetheless, it is incumbent upon institutions assessing prior learning to set 
appropriate program standards against which that learning is measured in keeping 
with accepted guidelines. The setting of prior learning standards at NLU is an 
ongoing process. The assessment program prototypes are continually scrutinized 
and periodically revised. However, one of the most crucial links in the evaluation 
chain, the evaluators of prior learning themselves, is perhaps one of the most 
neglected. To some extent at NLU, as elsewhere, evaluators and their 
qualifications are too often not considered a potential problem. Assumptions are 
frequently made that all evaluators should be faculty and conversely, that all 
faculty should be evaluators. As Whitaker points out, however, "all assessors 
(evaluators) should be experts, but not all experts, including teachers, necessarily 
make good assessors" (1984, p. 190). 

As a result of the present study's initiation, the NLU Assessment Center 
has begun to examine a number of salient questions, including: 

1. Who are our evaluators? 

2. Who should be our evaluators? 

3. How can we train our evaluators for excellence in performing their 
assessment functions? 

The current attitude of inquiry presents a challenge for the entire assessment staff. 

Importance of the Problem 

The Assessment Center is a vital yet alterative element of NLU's degree 
completion programs, inasmuch as formal recognition of prior educational 
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accomplishment is central to modem educational philosophy. The Center is 
expanding its services to include, for the first time, graduate students from the 
College of Education. Standardized CLEP and DANTES exams will be offered 
under its auspices beginning in October, 1991, and the feasibility of departmental 
challenge exams and interviews is being studied in an endeavor to revive NLU's 
proficiency program. As the sponsor of these activities, the Assessment Center 
continues to grow and develop, increasing and refining available options. 

Student, staff and evaluator frustrations continue, however. Although the 
Center Director has historically attempted to insure student satisfaction with the 
assessment process, this is sometimes difficuh due to discrepancies in expectations 
of various panies to the process. Students may be unclear on evaluator criteria, 
as may evaluators themselves; assessment counselors find themselves second- 
guessing both students and evaluators. Frequent expressions of these difficulties 
and misunderstandings make it quite clear that program values, assumptions and 
standards must be explicitly delineated to all those involved in PLA The task of 
programming for relevant training and development is a complex and difficuh one 
and the basis for this study. Specifically, what are the training needs of faculty 
evaluators and how can they be suitably addressed? This researcher will analyze 
the questionnaire responses of evaluators and assessment directors at NLU and 
other postsecondary institutions previously mentioned in order to ascertain quality 
assurance concerns, formulate a program of remediation and begin 
implementation through an evaluator workshop scheduled for October. 1991. 
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The Theory and Its Empirical Potential 

Theoretical Foundations 

Faculty roles > 'The college teacher potentially has an extraordinarily 
complex, rich composite picture of what a teacher looks like and does, or at least 
should look like and do" (Lacey, 1983, p. 96). However, the range of faculty 
responsibilities can vary in relation to the institution's values and priorities 
(French-Lazovik, \9S2). Along with many modifications in educational 
philosophies and systems have emerged questions regarding and challenges to the 
role of faculty in higher education* Methodological innovations derive from an 
attempt to better serve a new student population and consequently induce 
innovation in faculty function. Folger (1978) describes new faculty activities and 
new accountabilities for faculty efforts as direct outgrowths of such innovations* 
Educators today have the opportunity to "strengthen the faculty role . . . and 
contribute to both the survival and quality of higher education" (Loacker & Palola, 
1981, p. viii). 

Loacker (1981) characterizes the nature of the institutional change process 
as either diachronic (gradual, spontaneous change happening eventually through 
time) or synchronic (planned simultaneous change in support of a conscious 
decision)* Apart from any specific institutional approach to change, theories of 
contemporary teaching and learning have evolved through various forces and 
influences, broadening the faculty role from that of listener to recitation, to 
imparter of all knowledge, to orator, active learner, curriculum developer and 
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contributor to society (Bradley & Bolman, 1981). "Faculty members have 
gradually moved from the scholarly realm of teacher/researcher/evaluator into 
uncharted areas and are assuming a multiplicity of roles'* (Knapp, 1977, p. 2). 
Faculty are responsible for designing courses of study and defining degree 
requirements, competence levels and standards. In addition to facilitating learning 
and providing input on its goals and outcomes, faculty plan, implement and 
evaluate programs, analyze curricula and examine what methodologies work best 
for which purposes (Knapp, 1977). 

Dewey and the pragmatist school of thought, proponents of practical versus 
purely conceptual knowledge, see the faculty role as "one involving an articulation 
of the relationship between theory and practice" (Bradley & Bolman, 1981, p. 21). 
An emphasis on practical knowledge and alternative experiential learning. options 
have clearly influenced faculty teaching styles in drawing connections between 
theory and practice. The onslaught of adult clientele has again forced the 
development of new approaches and teaching styles as facilitators of "the 
transference from conceptual learning to real life situations" (Bradley & Bolman, 
1981, p. 25). Dressel and Marcus (1982) also define faculty as motivators and 
facilitators of learning, who themselves display a pervasive, yet directed curiosity. 
They participate with their students in a continuous learning process through 
"reflection on experience, both previous and subsequent, elaboration upon these 
reflections, and application of principles and generalizations derived from prior 
experiences" (Dressel & Marcus, 1982, p. 204). The passive character of many 
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classroom learning situations is cited by Freire (1972) as one of the chief 

drawbacks of traditional education. He portrays education in this sense as 

an aa of depositing, in which the students are the depositories and 
the teacher is the depositor (of knowledge). This is the banking 
concept of education, in which the scope of action allowed to the 
students extends only as far as receiving, filing and storing the 
deposits (Freire, 1972, p. 58). 

Rather than merely transmitting or "depositing" knowledge, however, faculty can 

be significant forces in the implementation of cognitive learning, as outlined in 

Bloom's Taxonomy of Educational Objectives, through comprehension, 

application, analysis, synthesis and evaluation. 

For some, adult and alternative learning signal enigmatic change in the role 

of higher education professionals. "Faculty will have a particularly difficult time 

maintaining their role as a result of recent changes in the larger society that have 

already had a significant impact on higher education" (Bradley & Bolman, 1981, 

p. 19). In reality, the current nontraditional movement, which Miller (1975) 

characterizes as personal counseling in combination with individualized learning, 

occasions a metamorphosis of traditional faculty thought and activities. 

The one aspect of future society that is clearly defined is change 
itself. Tomorrow's (educator) must be prepared to re-examine 
existing competencies and values and to alter them to meet the 
demands o.f accelerated change. Thus the educated person in the 
future must be flexible, a self-directed learner (Hoover, 1980, p. xi). 

Bradley and Bolman urge faculty to assume vigorous leadership in "raising 

questions about basic educational assumptions and traditional purposes of 

acquiring knowledge" in order to "reassess themselves as professionals and clearly 
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define their role" (1981, p. 28). 

Faculty development Hoover exhorts that "mastery of subject matter does 

not automatically qualify one for effective teaching" (1980, p. 335). Quality 

professionalism in the field cannot be achieved 

without an attack on the norm that teaching competence is innate 
and that it, therefore, need not be rewarded or be subject to hard 
and deliberate work .... Like all other scholarly pursuits, teaching 
must be open and subject to peer participation (Mauksch, 1981, 
p. 16). 

The art of teaching includes various components which are unrelated to 

content, such as educational process (methodologies and strategies), assessment of 

learning and evaluation skills. In addition, 

control over conditions of practice, access to appropriate resources, 
and the respect earned for being a teacher are basic ingredients as 
important to quality teaching as arc command of the subject matter 
and techniques of presentation (Mauksch, 1981, p. 15). 

Professional development, therefore, should be directed toward more effective 

teaching performance and is one means to the larger end of learning, not an end 

in itself. By nature it must be diagnostic, providing support and encouraging 

development and expansion of teaching methods and 'techniques within a 

framework of achieving positive results (Miller, 1975; Hoover, 1980). Educators 

are themselves adult learners, "learning professionals" with a need for 

systematically updating working knowledge and skills through self-development, 

structured exchange, and peer support (Cheren, 1979). A "commitment to learn 

how to aid the development of others along with the commitment to develop one's 
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own skills forms the nucleus of faculty development" (p* 22). 

Unfortunately, "faith in a connection between faculty development and 
improving teaching has not yet reached a point where faculty development 
receives a clearly defined and substantial amount of institutional support" (Eble, 
1983, p. 121). Eble does point out, though, that faculty development is generally 
being emphasized more than in the past Taculty development in a formal way, 
supported both by external and internal funds, is a modest phenomenon of the 
seventies" (Eble, 1983, p. 121). 

Despite changing administrational attitudes toward professional continuing 
education and the recognition that "how teachers learn on the job and how they 
continue to learn about teaching is one key to maintaining teaching excellence" 
(Eble, 1983, p. 120), all development efforts still depend upon the willingness of 
faculty to continue learning. Centra (1976) concluded from a survey of 756 
colleges and universities that "teachers who wanted to get better were the group 
most involved (in faculty development efforts) while those needing improvement 
were . . . least involved" (p. 59). Eble (1983) agrees with the conclusion that 
"there is probably no better way to drive faculty away from a program than to 
identify it as a service for the inadequate" (Centra, 1976, p. 59). Institutions must, 
instead, "create a climate conducive to learning and foster activities that result in 
learning" (Eble, 1983, p. 120). 

It is critical that this climate extends to beginning teachers, who need 
practical advice about teaching. Eble (1983) observes that, in this respect, 
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American universities may fall behind their British counterparts, since they require 
initial training programs during a staff member's first year, whereas American 
universities for the most part leave development of teaching skills to the 
individual Lacey (1983) views helping junior faculty to find their places in the 
profession and encouraging connections with appropriate role models and mentors 
as a major focus for any faculty development program, 

"But, there is still room for and need for keeping learning opportunities 
open for faculty members throughout their careers- Good teachers do continue to 
leam" (Eble, 1983, p. 131). And while professional development issues differ 
within organizations and in response to various individual developmental 
transitions, these issues impact "faculty growth, commitment, productivity, morale, 
and compatibility with the expectations of the university setting and . . . 
professional roles" (Cytrynbaum, Lee & Wadner, 1982, p, 11), 

Effective faculty development, i.e,, that which elicits the participation of the 
faculty most in need of development, is heavily dependent upon encouragement 
and support from all levels of administration. An additional invaluable element, 
according to Eble (1983), is a cadre of teachers recognized as outstanding which 
serves as a resource for the faculty development program. Furthermore, effective 
programs are generally low key and strictly voluntary due to the resistance of 
professors to being taught what they assume they already know. Lacey (1983) 
suggests that a successful faculty development program should focus on coalition 
building or networking, identifying important common interests that can best be 
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fulfilled by working with others. 

A politically astute program acknowledges, respects and plays to 
faculty strengths and should not be perceived as looking for 
weaknesses to overcome or errors to rid itself of. It assumes that 
faculty see therasehres as concerned, effective teachers and seeks to 

build support groups based on that foundation And a 

successful program should be pragmatic rather than ideological in 
emphasis. It is not a repository for right answers about effective 
teaching but a place to explore a faculty member's individual 

questions It will address the needs that teachers themselves 

bring forward (Lacey, 1983, p. 99-100). 

Even professionals, often enough, can underestimate their need for help 
and fail to realize that potentially useful resources are available. In the wider, 
more nontraditional view of the purpose of higher education, one of the most 
frequent obstacles to progress, however, is the absence of such an appropriate 
resource, according to Cheren (1979). "If teachers are to continue learning as 
teachers, there must be inescapable reinforcement of teaching, opportunities to 
develop in these directions, and rewards that attach to participation" (Eble, 1983, 
p. 135). In order for this to occur, "creating a climate in which teaching is a 
vibrant, satisfying, and ponderable activity may be the basic condition" (Eble, 1983, 
p. 140), thus maximizing the unifying potential of faculty development, with the 
craft of teaching becoming a "bond across departments, across disciplines, even 
across profoundly different styles" (Mauksch. 1982, p. 16). Kidd (1973) also sees 
the teacher as an artist and craftsman who "needs to take over some of the 
attitudes, such as the concern about skill, the devotion to self-improvement, the 
slow maturation of skill that is the hallmark of the genuine craftsman" (p. 297). 
Faculty development can nurture these attitudes and help teachers realize their 
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potentials. 

Experienti al learning theory . A most simplistic definition of this 

phenomenon is "learning by doing." Empirically, "it consists of what is given to the 

individual by his immediate environment and, cumulatively, by the sum total of all 

that he has learned" (Bills, 1985, p. 501). Simosko (1988) defines it as "learning in 

which the learner is in direct contact with the realities being studied or practiced 

to achieve a level of competence in a particular skill or knowledge domain" (p. 8). 

Experiential learning is lifelong and may be intentional or incidental, highly 

focused or seemingly almost at random. In essence "the term experiential learning 

. . . serves to describe legitimately most of the learning that occurs in our lives" 

(Simosko, 1988, p. 8), perhaps representing what Dewey calls "£he organic 

connection between education and personal experience (1938, pp. 19-20). Often 

diverse and highly individualized, it differs from classroom learning primarily in 

terms of educational input, rather than learning outcomes, and exists in a 

multitude of forms. Far from a new occurrence, 

today, more than ever before, educators are recognizing the 
importance of such sayings as "there is no substitute for experience" 
and "experience is the best teacher." The world of experiential 
learning is being described, explored, and mapped in ways that will 
enable academic institutions to utilize this kind of learning more 
effectively (Nesbitt, 1977, p. 2). 

Experiential learning is based on the precept that knowledge is necessary 
on two complementary levels: 1) informational competence (conceptual grasp or 
KNOWING) and 2) practice (behavioral performance or DOING) (Duley & 
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Gordon, 1977). The significance of experience or practice in the educational 
process has been elucidated across generations of twentieth century educators 
such as Lindeman, Bloom, Freire, Kolb and Coleman, 

Linderaan and his pragmatic-progressive colleagues advocated many 
concepts shared by modem day adult educators as outlined by Stewan (1987), 
including: 

the centrality of the learner's exf)erience, learner growth as the basic 
aim of education, a process orientation rather than a goal 
orientation, problem emphasis rather than a subject emphasis, . . . 
and teacher as facilitator (p, 140), 

Lindeman's fundamental assumption that the resource of highest value in adult 

education is the learner's experience reflects the influence of Dewey and is later 

mirrored in Knowles' (1968) premises for andragogy, the an and science of 

teaching adults (Stewart, 1987), It was Lindeman who prophesized the leap of 

adult education from the restraining boundaries of educational institutions to 

become integral with life and work activities" (Sheckley, 1987, p. 6). 

Bloom (1965), as related earlier in this text, describes the activity of 

thinking in a hierarchical order beginning with awareness and proceeding through 

knowledge, comprehension, application, analysis, and synthesis to evaluation. 

Ideally we are seeking a problem which will test the extent to which 
the individual has learned to apply the abstraction in a practical way. 
This means that the problems should have some relation to the 
situations in which (the student) may ultimately be expected to apply 
the abstraction (Bloom, 1965, p, 125), 

Freire (1972) proposes that education for freedom enhances human dignity 

through dialogue and ends in action. The learner modifies his basic perspectives 
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on reality through a critical, active process. 

Apart from inquiry, apart from the praxis, men cannot be truly 
humaiL Knowledge emerges only through invention and reinvention, 
through the restless, impatient, continuing, hopeful inquiry men 
pursue in the world, with the world, and with each other TFreire 
1972, p. 58). ^ 

Similarly, Coleman (1976) differentiates the experiential learning mode 
from the traditional (information assimilation) based upon the active nature of the 
process and the source of information. Experiential learning is more inductive in 
nature, with information coming from acting or observing action, versus traditional 
deductive learning from symbolic sources such as lectures or reading. In 
Coleman's model, experiential learning emanates from: 1) information from 
action and observation, 2) applying information in specific instances, 3) 
understanding general principles and 4) re-applying general principles in new 
situations. 

The experiential learning theory of Kolb (1984), too, describes adults as 
active learners who shape the reality they perceive rather than passively receiving 
it Building on other theorists such as Dewey, Lewin and Piaget, Kolb has 
postulated that learning involves a cycle of four processes. 

Ideas or skills are grasped through concrete experiences or abstract 
concepts. F.or learning to occur, the ideas or skills must also be 
transformed, through reflective observation or active 
experimentation, to fit into an individual's existing perspectives or 
capabilities (Sheckley, 1986, p. 11). 

Brookfield (1987) particularly emphasizes the reflective process as essential to 
adult development and learning, which demands relating new situations to 
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previous ones and contemplating alternative ways to thinking. 



Thus while experience is seen today as the sine qua non of learning, it is 



widely recognized that "experience alone does not generate new learning; 
(however,) neither does the college classroom" (Sheckley, 1988, p- 176). A new 
interplay has unfolded between various educational providers, and its impact on 
tradition has been immense. Institutions have incrementally extended and re- 
interpreted the very meaning, though not the aims, of higher education, 
eliminating or at least making more difficult to discern some of the acute 
distinctions between traditional and nontraditional education (Kjiapp, 1977). 

"College and university teachers have too long assumed that the important 
activities of a course are those that go on in the classroom under their direct 
supervision" (Dressel & Marcus, 1982, p. 208). Cross (1979), though, refers to our 
culture as "the Learning Society," which, due to factors such as escalation of the 
rate of change, demographics, and increased competition in the labor market, 
consists of a rich mixture of learning resources and in which colleges and 
universities provide only about a third of the organized instruction available for 
adults. 

Schools no longer have a monopoly on education ... no educational 
provider has any monopoly on the learning market. Clearly, 
education for adults has burst explosively from its physical 
boundaries, and learning is now acknowledged to be in the 
individual rather than in the buildings and professors of the ivied 
halls. Once learning is perceived as a characteristic of the learner 
rather than an offering of the provider, attention is shifted from 
teaching to learning. It is that shift that will revolutionize education 
(Cross, 1979, pp. 8-9). 
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An expanded view of what types of knowledge are worthwhile within an 
academic framework has produced questions of how that knowledge can or should 
be transmitted and assessed. The nature and means of acquisition of experiential 
learning in contradistinction to classroom learning "require the consideration 
of . . . techniques that have not been widely employed in higher education (but 
which) . . . should have corresponding benefit for higher education generally" 
(Knapp & Sharon, 1985, p. 5). Methods of teaching and assessing which "take 
fully into account the individual character of students and their learning 
experiences" (Knapp & Sharon, 1985, p. 5) are certainly relevant in any 
educational setting. The intellectual inquiry into the meaning of knowledge and 
education and learning has always revolved around the assessment or evaluation 
of that learning, which is "at the heart of the educational process at any level and 
in almost every context" (Simosko, 1988, p. xii). In the case of experiential 
learning, the "burden of proof lies more heavily with the learner to describe and 
document learning outcomes, but also with the faculty evaluator in terms of 
measuring and comparing those outcomes to college level expectations and 
criteria. 

Experiential education . In a broad sense, experiential education is an off- 
campus, nonclassroom learning activity which may take place under the auspices 
of an educational institution (sponsored experiential learning) or independently 
thereof, usually prior to matriculation (nonsponsored or prior learning). While 
some forms of experiential education are well-known through many years of use, 
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others are newer and less familiar. 

Sponsored experiential learning includes such diverse offerings as cross- 
cultural experiences, cooperative education (work-study programs), 
pre -professional training, preparatory internships and apprenticeships, social and 
political action opportunities, personal growth and development programs, career 
exploration, field research activities and career or occupation development 
placements. These types of educational experiences have been designed in 
response to "the educational goals of (today's nontraditionai student which) extend 
beyond the normal goals of the traditional educational system" (Duley & Gordon, 
1^77, p. 2). The sponsored experiential learning environment supplements 
classroom learning by providing "opportunities for application, analysis, synthesis, 
and evaluation of knowledge (required for) the full development of one*s potential 
in the cognitive domain" (Duley & Gordon, 1977, p. 6). While college classrooms 
and labs provide students with theoretical background, accumulation of 
information and essential skills, college-sponsored field work affords the 
opportunity to apply this learning in practical situations, thus making the learning 
truly relevant (Nesbitt, 1977). 

Nonsponsored or prior learning of what is studied in a particular college 
course occurs in personal, professional, and avocational life. It may ensue from 
internal interests or more practical needs. Sources include life accomplishments, 
work and careers, hobbies, community service and self-directed learning projects. 
"In recent years there has been a significant increase in the kinds of areas that are 
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assessed for credit and, therefore, in the kinds of students who are being 
encouraged to use the various credit-bearing options" (Simosko, 1988, p. 64). 
Cabell and Hickerson (1988) note that a number of ways have been developed for 
ascertaining prior learning (e.g., placement tests, proficiency exams, evaluation of 
licenses and certifications, portfolio development), and a variety of program 
models have been established for translating assessments into academic credit. 

Assessing experiential learning . Whether assessing sponsored or 
nonsponsored experiential learning, 

institutions face a variety of problems in introducing procedures for 
assessment of experiential learning . . . and implementing new 
assessment techniques into existing administrational procedures, 
academic policies and organizational structures (Willingham & 
Nesbitt, 1976, p. 1). 

Assessment is complicated by the fact that the learning has usually not been 
acquired under direct supervision of faculty and may involve competencies not 
typically included in comparable college courses. The unanticipated learning and 
uncontrollable constraints typical of experiential learning situations challenge the 
assessment process and the people who attempt to evaluate learning outcomes 
(Duley & Gordon, 1977). 

Prior learning assessment, in particular, while a viable option for numerous 
adult students who would otherwise not be afforded recognition of their 
accomplishments (Cabell & Hickerson, 1988), presents certain discrete challenges 
for evaluators. By definition, PLA addresses aU of an individual's college-level 
learning, no matter when or how it was learned. Yet the distinction between mere 
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experience and actual learning is crucial in assessment. While personal life is an 
important potential learning environment, 

k is clear that there is no necessary correlation between experience 
and learning. It is important to recognize the difference between 
experience that is relatively easy to describe and to measure and 
learning that is relatively difficult to assess (Breen, Donion, & 
Whitaker, 1977, p. 40). 

The PLA process is further "complicated by the personal nature of the 
portfolio, which assumes that no two portfolios can ever be the same" (Cabell & 
Hickerson, 1988, p. 149). In this assessment mode, evaluators constantly face 
differing situations, extenuaimg circumstances and uncontrollable variables. There 
exist few steadfast regulations, and flexibility must be the norm. Institutional 
guidelines for assessment practices are needed in order to offset these 
compounding factors insofar as possible. 

Because of the complex factors and uncontrolled environment involved in 
experiential learning, it is more difficult for students to recognize and articulate 
what has been learned. Therefore, the assessment process must assist students in 
reflecting upon and organizing their learning. In order to accomplish this, the 
process should be open to unexpected learning in addition to the stated objectives 
and should have educational value of its own. Formative assessments are one way 
of achieving this through educative evaluator feedback. The process itself, besides 
promoting reflection and self-assessment, serves as a foundation for effective 
future learning. "Evaluation in order to develop greater congruence between 
goals, objectives, and outcomes is what is needed" (Duley & Gordon. 1977, p. 51). 
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Simosko (1988) maintains that although the process of assessing 
experiential learning is different from the process of assessing classroom learning, 
the same kinds of academic judgements about what a student knows and can do 
are required. Hence, expected learning outcomes must be defined and criteria for 
success clearly demarcated, as in classroom practice. However, experiential 
learning assessment "must focus only on the outcomes of the learning" (Simosko, 
1988, p. 63), rather than inputs evaluators might ordinarily provide in classroom 
instruction. The specific assessment method must be appropriate to the subject 
matter under evaluation and to the products, performances, and external 
documentation presented as evidence of learning. Learning must be equated to 
that expected for a comparable college-level curriculum. "Assessment is about 
generating evidence and making judgements of an individual's competence, by 
comparing his or her performance against the established criteria" (Mitchell, 1987, 
p. 2). While there exists no "absolute standard" for assessment, it is essential for 
evaluators to strive for a high level of reliability. 

Evaluator development . Cheren (1979) examines the effect on faculty 

development of the rapidly increasing proponion of middle aged and older adult 

learners and finds that the prevalence of more innovative practices and increased 

focus on individualized and experiential learning necessitates professional growth. 

Meeting the needs and preferences of returning students requires an 
approach to facilitating the learning of others which is different, 
more difficult, and more demanding than traditional methods 
required to achieve the same ends" (Cheren, 1979, p. x). 
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Cabell and Hickerson (1988) agree that thoughtful development efforts 
should be launched to "help faculty work with the increasing numbers of learners 
with wide ranges of abilities, interests, and life experiences" (p. 132). While 
continuing professional development is basic to lifelong learning for all faculty, 
those involved in more innovative practices such as assessment of experiential 
learning face an even greater imperative to continue to learn in the field (Cheren, 
1979). Kidd (1973) acquiesces that skill development is fundamental to the 
continuing education of teachers, noting that specific teaching styles and roles may 
require special kinds of skills, few of which are easily acquired without some 
effon. 

The role of experiential learning evaluator is one such instance requiring 
special skill and effon. 

The individualized and often highly unique nature o'' experiential 
learning poses a number of problems for the faculty members and 
administrators charged with the assessment of that learning .... 
Assessment must, therefore, be placed in the hands of an "expert" 
(Reilly, 1977, p. v). 

Reilly (1977) stresses that these "expens" must be properly prepared to conduct 
assessments of learning, whether derived from sponsored or nonsponsored 
experiences, in order to assure equity both for the institution and the student. 
Preparation for this specialized form of assessment cannot be ignored where 
quality assurance is at stake. It is not reasonable or prudent to immerse 
traditional faculty in the experiential learning arena and expect them to function 
with only familiar techniques useful in a classroom setting. "Assessors need to 
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learn their jobs and strive to do them consistently and fairly over time .... 
Decisions . . . need to be right and to elicit agreement among other experts on a 
regular basis" (Simosko, 1988, p. 65). 

In the early days of prior learning assessment, little attention was given to 
formal preparation of faculty evaluators, despite the fact that "faculty assessors are 
at the heart of every PLA program, and selecting and training them is an ongoing 
process that is critical to the program's success" (Craig, 1990, p. 19). 
Unfortunately, this situation still prevails all too often, with perhaps the exception 
of initial faculty orientation. 

The first step in evaluator development is so obvious that it is often 
overlooked, but "the task of choosing appropriate assessors (evaluators) is a 
critical and difficult one" (Reilly, 1977, p. 2). As noted previously, "all assessors 
should be experts, but not all experts, including teachers, necessarily make good 
assessors" (Whitaker, 1976, p. 190). Evaluator recruitment ought to be viewed as 
a weighty issue with ramifications for students and assessment staff alike. 
Sometimes assessment is a mandatory faculty function, while at other institutions it 
is a voluntary "extracurricular activity" for regular faculty. In some cases, 
additional hiring may be needed to properly staff a PLA program, or release time 
arranged for existing personnel (Willingham & Nesbitt, 1976). 

Whitaker (1976) offers useful criteria in evaluator selection which he labels 
"essential assessor characteristics": 

1) subject matter expertise (appropriate to the specific learning being 
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claimed) 

2) psychometric expertise (knowledge of the assessment process and 
appropriate techniques and instruments) 

3) familiarity with case data (all sources of information about learning 
outcomes) 

4) objectivity (unbiased judgment based on the evidence) 

5) motivation (for thorough and equitable assessment, and to maximize 
qualifications as experts in subject matter and in assessment) 

Interestingly, these "essential assessor characteristics" parallel closely David and 
Parker's (1979) suggested characteristics of the ideal doctoral candidate advisor, 
especially with regard to competence, adherence to standards and integrity. 
Teachers beyond the classroom situation apparently face many similar personal 
requirements in the interest of maximizing learning possibilities. 

Although not ail evaluators will possess each characteristic, "careful 
identification and selection of expert judges can considerably simplify the task of 
preparing assessors" (Reilly, 1977, p. 4). Ultimately, PLA directors will decide 
what specific evaluator characteristics and qualities are required or desirable for 
their programs and then recruit appropriate faculty by finding ways to make 
panicipation appealing to them. Such "rewards," according to Craig (1990), could 
include release time from other duties, awards or recognition days, or research 
opportunities. For some, the opportunity to work with a different student 
population might be sufficiently appealing. 
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As a rule, few colleges spend much time or money on helping faculty 
assess learning. Initial training offered to faculty in many assessment 
programs is often the only exposure faculty have ever had to good 
assessment (Simosko, 1988, p. 69). 

Informal tutoring by program advisors is probably the most commonly utilized 

faculty training method, in conjunction with occasional seminars and workshops; a 

group or team approach is seldom used unless a faculty member requests specific 

assistance from colleagues or the subject matter is of an interdisciplinary nature 

(Cabell & Hickerson, 1988). 

Simosko (1988) remarks that, as subject area specialists, faculty are 

assumed to automatically know '\vhat students should know and how best to 

determine whether a particular student has achieved the expected learning" 

(p. 69). However, basic concerns in assessment must be studied, requiring a new 

set of professional knowledge and skills. Even if evaluators feel sufficiently adept 

at specific programmatic practices, "there is still a need to become conversant with 

assessment techniques, methods and knowledge outside (the) program or 

institution and to apply this awareness , , , within (the) institution" (Knapp, 1979, 

p. vii). 

Development of faculty resources, however, is one of the most important 
components of the experiential learning program, including faculty and staff 
training in PLA techniques. This component bears upon all other aspects of the 
program, suggesting a mutually def)endent relationship within the overall system 
(Willingham & Nesbitt, 1976). Such issues as the extent and character of training 
needed, the number of staff needing such training, faculty resistance to training. 
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clarification of faculty responsibilities and the relationship between nontraditional 
and traditional academic programs and faculties need to be addressed. 

Whitaker (1989), in what has become "the Bible of prior learning 
assessment," sets forth ten academic and administrative standards for assessment 
with underlying principles and consequent procedures, two of which are 
particularly relevant to this discussion: 

Standard IV - The determination of competence levels must be made 

by appropriate subject matter and academic expens. 

Standard X - All personnel involved in the assessment of learning 

should receive adequate training for the functions they 
perform, and there should be provision for their 
continued professional development. 
Even for those faculty with expertise in classroom learning assessment, 
professional development is essential in suppon of experiential learning 
assessment "Research indicates that judgments of different assessors can vary 
considerably and systematically if they have had limited experience or if 
assessment guidelines are not clear'' (Whitaker, 1989, p. 60). Evaluators need to 
understand experiential learning and be proficient in methods for its assessment, 
writing evaluations, interpreting standards, and enhancing accuracy of assessment 
(Whitaker, 1989; Willingham, 1977). Whitaker (1989) and Willingham (1977) 
further suggest the importance of supervised experience for new evaluators in 
assessing learning and awarding credit so that their emerging personal standards 
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will more closely align with those of the institution. Written guidelines and faculty 
evaluator training activities are necessary to achieve and maintain consensus on 
assessment issues. 

The role of the faculty evaluator in PLA is similar to some extent to that of 
the faculty advisor in a sponsored experiential learning program. In both 
instances, the primary concern is to appraise the experience and to assess what 
has been learned from it Both evaluator and advisor provide feedback, help 
frame the proper questions for interpreting experiences and, ideally, suggest new 
•vays to think about a particular problem or experience (Nesbitt, 1977). 

Special problems are associated with prior learning, however. Often it 
cannot be equated to traditional course offerings. The educational rationale may 
even differ from that of traditional programs. Measurement techniques vary in 
applicability to various types of experiential learning, and evaiuators need to know, 
choose and be able to apply appropriate techniques, such as essays, product 
assessment, inte»'Anews, oral exams, simulations and performance assessment. In 
addition to identifying learning outcomes like the faculty advisor in a sponsored 
situation, the evaluator must determine whether the learning acquired in a 
nonsponsored setting is college level. 

Reilly (1977) concludes that even "expert judges are often quite fallible" 
(p. 6). Assessment necessarily involves a certain amount of error, either 
systematic or random (Willingham & Nesbitt, 1976). While systematic error 
involves a consistent bias in a particular direction and is wrong "consistently, 
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dependably, predictably and reliably" (Willingham & Nesbitu 1976. p. 26), random 
error occurs by chance and with fluctuations due to unknown causes, thus resulting 
in decreased reliability in assessment Although "it should be recognized that 
absolute consistency of assessment can never be attained . . . clearly the 
consequences of incorrea judgements will need to be considered and a balance 
struck" (WiUingham & Nesbitt, 1976, p. 24). 

Fortunately, assessment experts agree that it is possible to correct or 
alleviate many of the errors evaluations make. One way to do this is to make 
standards more explicit and objectively defined. Evaluators will not always regard 
an assessment in precisely the same way, but the foundations from which they 
proceed must be identical; this postulate is central to faculty evaluator 
development 

Since many of the problems with judgmental assessment stem from 
individual assessors operating with different standards, a goal of any 
institution wishing to improve its assessment should be to ensure 
that assessors are operating within a common framework of values 
and criteria (Reilly, 1977, p. 18-19). 

Cabell and Hickerson (1988) also urge that clearly articulated standards and 

well-monitored procedures, along with competent evaluators and sound assessment 

processes, are needed to maintain quality in evaluation. 

In addition to periodic reviews of assessment results, evaluator training 

sessions are elemental in promoting consistency in the application of assessment 

standards. "The greater the number of faculty . . . involved in the evaluation of 

learning, the harder it will be to achieve and maintain consistency" (Willingham & 
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Nesbitt, 1976, p. 24). Consequently, evaluators need to understand the kinds of 

learning being evaluated and the levels of evidence required before credit can be 

awarded. Refinement of standards of acceptable performance and achievement 

for academic recognition, both for classroom-based and experience-based learning, 

is mandatory for trustworthy and equitable educational assessment. 

Sound standards are central to the credibility and acceptance of 
credit for experiential learning. When carefully developed, 
appropriately applied, and closely monitored, standards assure the 
student, the school, and the public that credit awarded is credit 
awarded for college-level learning, and that the amount of credit is 
appropriate and fair (Knapp & Jacobs, 1981, p. 24). 

How, then, can evaluator development promulgate accurate measurement 
and evaluation? Simosko (1988) finds that most training programs for faculty 
evaluators focus on the traits of adult learners and how they may differ from 
"traditional" college students. They also help faculty begin to think in terms of 
learning outcomes, standards, and alternative assessment methods. Evaluator 
development programs can assist faculty "to evaluate as well as value the different 
strengths individuals possess" (Simosko [Ed.], 1988, p. 23). 

Evaluators need, and training programs can provide the opportunity to 
network with others skilled in the assessment process, a decisive factor in 
evaluator success. 

... It is rare to find a learner who does not need advice and 
prodding or at least the loan of a friendly ear from a peer, a mentor, 
or a group of professionals during the learning process. Educators 
offering individual study programs have long noted that considerable 
contact with other learners, mentors, facilitators or advisors is 
requested frequently by the (adult) learners in their programs . . . 
there is little reason to suspect it will be any different for (faculty 
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evaluators) (Knapp, 1979, p, vi). 
Craig (1990) recommends group training sessions with participation by 
experienced "model" evaluators and influential administrators who support and are 
committed to PLA In addition, outside experts with good teaching skills are 
highly desirable for conducting faculty training in conjunction with resident PLA 
directors and staff. Workshops, conferences and seminars, on or off campus, are 
typically offered in support of teaching and are useful in support of PLA as well. 
Lacey (1983) suggests that familiar types of low-risk activities minimize a 
perceived threat while reinforcing commitment to professional activities and 
encouraging participants to broaden their own understanding of significant 
learning and assessment issue s. Forrest, Knapp and Pendergrass (1976) admonish 
that 

training workshops and appropriate materials should be developed 
to stimulate creative thinking in selecting and designing 
measurement techniques for many types of learning outcomes, to 
improve the capabilities of assessors in writing concise and 
appropriate evaluations . , . and to increase the reliability of the 
judgment of assessors who must make decisions regarding the actual 
awarding of credit based on a ponfolio process (p. 177). 

Knapp (197?) further posits that while peer and group work in a workshop 

or seminar setting is beneficial for some learners and for some learning objectives, 

not all objectives and activities for evaluator development need to be bound to 

that type of instructional setting. Craig (1990), for example, identifies a need for 

exposure to the literature of adult education and theory which can be approached 

by means of regular handouts or newsletters apart from a group setting, Whitaker 
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(1989) states the desirability of having appropriate instructions and reference 

materials, including a faculty handbook, available to evaluators at all times for 

individual use^ And Lacey (1983) condones individual consultations with experts 

by faculty request on various relevant aspects of learning. 

Whatever the approach to evaluator development, there can be no doubt 

that quality prior learning assessment is impossible without iL In a territory full of 

different and unfamiliar concepts, faculty are obliged to consider the issues 

involved in the ' ;velopment of a new realm of expertise. Each institution active 

in PLA needs to weigh its own particular needs and design a means of meeting 

those needs, for the sake of its students, its faculty and the very integrity of the 

institution itself. 

Thus, training programs offer an exciting opponunity tor 
professional development, reflection, and intellectual inquiry into 
what is really important in a particular field of study. Faculty who 
learn . . . will find the assessment process professionally satisfying 
and enriching to their work as teachers, evaluators, and ultimately, 
of course, adult learners (Simosko, 1988, p. 69). 

Relevant Evidence 

The number of empirical studies in the realm of faculty evaluator 
development per se is limited, perhaps because of the difficulty in collecting "hard 
data" in the field. Experts agree unanimously that development must be 
undertaken, but few have carried out studies on the effectiveness of specific 
methods and activities, or even on the distinctive training needs that exist. 
Perhaps, too, the fact that this domain has not historically been highly valued in 
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part accounts for the scarcity of relevant empirical studies. 

The mid to late 1970s yielded the majority of work done on strategies for 
assessing prior learning. The GAEL Validation Report (1976) was originally 
conceived in the fall of 1973, as assessment was becoming a major priority in 
higher education. It was a developmental work, but also incorporated a series of 
validation studies to begin to address questions raised by the new interest in 
crediting prior learning, such as adequacy of assessment procedures, their equity 
and educational soundness, protection of students and academic standards, and 
the role of institutions as credentialing agents. Basic data tor the field research 
was collected at 24 sites. The GAEL project was organized in three overiapping 
stages. The development stage involved production of new assessment procedures 
and materials. In the second stage, validation, the products of the development 
stage were tested and improved Thirdly, in the utilization stage, the previous 
work was made available to professionals concerned with PLA. 

Two conclusions of the Report, in particular, are relevant to the issue of 
faculty evaluator development. '*One implication is a need for better institutional 
guidelines regarding assessment practices; another is a need for taculty 
development programs — a need that seems likely to continue for some time" 
(Willingham, 1976, p. viii-7). 

Both conclusions relate to a weakness in portfolio assessment that may 
ensue from the absence of a well-trained evaluator resource. "Results suggest that 
there is danger of fairly substantial discrepancies in credit awards if faculty judges 
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are inexperienced" (Willingham, 1976. p. viii-S). Inexperienced or less experienced 
evaluators may make radically different credit awards because of differences in the 
assumptions they make about what learning is creditable, what might overlap with 
other learning, whether learning is current, and so on. These assumptions need to 
be uncovered and discrepancies readdressed through discussion of policies and 
guidelines. According to WUlingham (1976), this should be done in the initial 
evaluator development phase, and then periodically for purposes of quality 
control. 

In 1975 and 1976, a small site visit study wad conducted at 80 GAEL 

member institutions using PLA. In this study, once again, 

. . . findings point clearly to a need for placing the skills needed for 
assessment and planning and implementation of programs in the 
hands of those individuals who are responsible for maintaining 
academic quality — the faculty. Workshops or seminars could fulfill 
this need . . . (Knapp, 1977). 

In response to this finding, the GAEL Faculty Development Program was 
begun using a "train the trainers" model. GAEL originally trained 12 facuhy teams 
from various institutions throughout the country in assessment skills and strategies 
for changing resistant attitudes toward experiential learning. Each team then 
trained 10 to 12 other teams on a regional basis, with the second wave of trainees 
conducting workshops on their own or nearby campuses. Again, two conclusions 
from this program impact the present study. First, faculty workshop programs 
should emphasize "hands on" experience with assessment problems. Secondly, "if 
workshop models are not possible, frequent faculty meetings or an apprentice 
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system should be encouraged'* (Knapp, 1977, p. 18). 

The Commission on Educational Credit of the American Council on 
Education appointed a Task Force on Educational Credit and Credentialing in 
1974. The ensuing two-year study was funded by a grant from the Carnegie 
Corporation. The Task Force deliberated on and analyzed problems and issues, 
producing working papers and in 1977, a final report endorsed by the ACE Board 
of Directors, including a set of 15 recommendations for improving recognition of 
educational accomplishment and credentials. 

Due to the importance attached by the Commission to assessing learning 

attained in extrainstitutionai settings, researchers concluded that qualified faculty 

should predominate as expert evaluators. Since few faculty have been formally 

trained in PLA, "with only moderate effort devoted to improving their evaluation 

skills," (Miller & Mills, 1978, p. 233) the Task Force suggested 

faculty development programs on evaluation as one means for 
improvement, and the inclusion of evaluation competencies in 
graduate programs for preparing new facultv members as another 
(Miller & Mills, 1978, p. 233). 

A related recommendation reiterated that awarding of credit for 
extrainstitutionai learning should fall under the auspices of faculty. Therefore, the 
Task Force directed, in line with previous CAEL findings, that "increased attention 
to evaluation procedures and techniques is necessary in the assessment of 
nonsponsored prior learning and that faculties are obligated to be informed about 
policies and procedures" (Miller & Mills, 1978, p. 234). 

Knapp and Jacobs surveyed 330 CAEL member institutions in 1979, Their 
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purpose was twofold: 1) to determine what was being done in setting PLA 
standards and 2) to determine what should be done in setting those standards. 
Practices were ascenained from a 65 percent response rate, with respondents 
identifying major and alternative methods used for awarding credit for prior 
learning. 

One important finding from this study was the "disappointingly high 
proportion" (Knapp & Jacobs, 1981, p. 9) of institutions awarding credit on the 
basis of experience rather than learning. Funhermore, the researchers concluded 
that many institutions rely excessively on the expert judgement of faculty without 
the benefit of a consensus about PLA standards. While a majority of programs at 
that time provided evaluators with general assessment guidelines (79 percent), 
only 38 percent offered professional development workshops, and a mere 27 
percent made available an assessment handbook as an aid to evaluators (Knapp & 
Jacobs, 1981). 

A new CAEL institutional survey is currently under way as part of a joint 
project with the American Council on Education to improve PLA programs. Data 
on assessment and evaluation practices was collected during the summer of 1991; 
a preliminary report of this study indicates a 47 percent response rate, with 1,732 
completed surveys returned from the 3,694 schools contacted. Recommendations 
and an action plan can be expected to yield similar concerns and suggestions in 
regard to the paramount issue of faculty evaluator development in an overall 
climate of quality assurance. 



88 



78 

Need for Study 

Role of the Thesis 

An increase in the adult college student population and the resultant 
flourishing practice of awarding academic credit for prior experiential learning has 
resulted in a need for faculty expertise in the technical aspects of prior learning 
assessment Program staff must be aware of and sensitive to these needs as the 
primary providers of appropriate training and development opportunities. Quality 
assurance in PLA is largely dependent upon faculty evaluator preparation and 
performance. 

From selection to initial orientation to continuing professional education, 
decisions regarding faculty evaluators influence the academic integrity of any PLA 
program. Administration along with faculty must take responsibility for 
ascertaining training and development needs, outlining specific objectives and 
addressing these factors in an appropriate manner, PLA has gained recognition as 
a valid educational alternative, but a review of the literature has disclosed an 
urgency in maintaining that validity through attention to criteria and standards in 
assessment. 

This thesis will help to identify some specific professional development 
needs, an objective seemingly lacking in most previous studies of assessment 
practices. Administrator and evaluator input will reveal personal perceptions and 
attitudes pertaining to PLA Empirical data on the assessment process is liiuited 
by its very subjective nature and by the considerable number of uncontrollable 
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variables inherent in such a quantitative educational topic. Nonetheless, 
awareness of the relationship between training and execution along with an 
understanding of the theoretical basis for excellence in evaluations will be 
beneficial to the entire assessment staff. 

Potential Impact on the Thesis 

This study has the potential to affect and ultimately improve the PLA 
process both directly through progressive evaluator training, and indirectly through 
consequently enlightened assessment counseling practices. The parameters of this 
research extend beyond quantitative constants into the affective perceptions and 
needs of a specific group of evaluators. Assessment staff will be able to utilize 
these findings to design and implement faculty evaluator training programs as one 
means to the end of improving prior learning assessment practices. 
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METHODS, RESULTS, INTERPRETATIONS, AND RECOMMENDATIONS 

Methods 

Design 

This study utilized a multi-pronged, comparative approach to identify 
training and development needs specific to faculty evaluators of prior learning 
experience essays. It emphasized both problematical and remedial aspects of 
prior learning assessment in regard to faculty roles as part of a larger effort to 
develop a more comprehensive program of continuing professional development 
for NLU faculty evaluators. 

Subjects 

Two discrete populations were surveyed in the course of the study, the first 
of which was comprised of internal or institutional (NLU) data sources. 
Preliminary and follow-up interviews were conducted during the summer of 1990 
with Cynthia Scarlett, Director of the Center for Educational Systems and 
Services, a unit providing consulting expertise in PLA to contract institutions. In 
addition, Ms. Scarlett previously served as director of NLU*s Assessment Center. 
Her work with multiple educational institutions enables her to experience 
problems and solutions from a more pluralistic perspective. 

Thirty-nine active NLU faculty evaluators were identified and surveyed, 
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with 30 evaluators (77%) completing and returning questionnaires. Some of these 
evaluators had up to 12 years of experience in PLA and have completed hundreds 
of essay evaluations, while others were much newer to the process and relatively 
inexperienced The current NLU Assessment Center Director was also asked to 
participate in the study by completing the Evaluator Training Survey for 
Assessment Directors. 

The second population included in this study was external or extra- 
institutional and consisted of 14 directors of PLA programs; eleven directors 
(79%) responded. Six directors were selected from colleges which have contracted 
with NLU's Center for Educational Systems and Services, with a response rate 
among this group of 100%. Eight directors from other postsecondary institutions 
with well-known PLA programs were also contacted, but only 5, or 63%, returned 
questionnaires. Additionally, an independent assessment of NLU's degree 
completion program by Urban Whitaker of CAEL is reviewed for the purpose of 
abstracting appropriate recommendations. 

Instrumentation 

A personalized cover letter was sent to each participant explaining the 
study's purpose and .intent. Each also received the appropriate questionnaire, 
depending upon professional status. Faculty evaluators received a 24-item 
questionnaire, while assessment directors received a similar albeit somewhat 
modified questionnaire containing 25 items. Both surveys were based on the 
Research Questions explicated in Chapter One of this study and included several 
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queries of a demographic nature. While the majority of items allowed a multiple- 
choice response, participants were also given an opportunity to provide expanded 
feedback and personal comments. Surveys were color, coded for the two groups. 

An open-ended interview based loosely on the surveys and the findings of 
an expert program evaluation have served as additional sources of information for 
summarization in the present study. 

Procedure 

After the list of panicipants was finalized in consultation with the directors 
of the Assessment Center and the Center for Educational Systems and Services, 
cover letters and surveys were mailed to faculty evaluators and assessment 
directors. Subjects were asked to complete the surveys from their respective 
viewpoints, circling all applicable responses and writing in comments where 
pertinent. A return deadline was noted on the questionnaires to encourage timely 
completion. Due to time constraints and a fairly healthy response rate, non- 
respondents were not contacted at the deadline date as originally planned. 

Data was transformed to numeric coding and processed via IBM-PC by the 
Statistical Package for Social Sciences (SPSS). Category data was derived from 
the study variables and measured on nominal or ordinal scales, and nonparametric 
statistical techniques were applied. Frequency distributions were the primary data 
summaries generated in response to research questions. 
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Results 

Demographic Characteristics 

Among NLU evaluators panicipating in this study, 40% are full-time 
university faculty members, and another 40% are adjunct or part-time faculty 
members. The remaining 20% of respondents categorize their positions as outside 
experts employed specifically for the purpose of PLA. The evaluator status 
distribution is illustrated in Figure 5, 

Fi gure 5 . NLU evaluator status distribution. 




Extrainstitutionally, 100% of assessment directors indicated that faculty 
members are most often involved as evaluators in PLA Fifty percent include 
assessment staff members in the evaluation process, while 25% of the programs 
utilize faculty committees in addition to the other personnel. As at NLU, 20% of 
directors also employ outside experts as evaluators. 

Nearly half (40%) of the NLU evaluators report having been recruited for 
PLA by assessment staff members, compared with 27% who responded to a 
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general invitation to faculty. Relatively few (23%) e valuators have been 
specifically recommended for that role, either by their department heads or by 
fellow evaluators. 

Assessment directors at the institutions surveyed also identify staff 
recruitment as the primary means of selecting evaluators; 82% employ this tactic. 
However, 64% also recruit through general faculty invitation, while 45% take 
advantage of recommendations. 

For the purpose of determining degree of activity, evaluators and directors 
were asked to estimate the average number of assessments performed over a one- 
year period NLU evaluators disclose a wide variance in numbers of assessments 
(see Appendix M), although the majority perform from 11 to 50 per year; this 
category appears to be the norm for evaluators at the other institutions as well. 
No directors report evaluators averaging in excess of 100 evaluations per year. 

The great majority of NLU respondents have served in PLA for a 
considerable length of time: few inexperienced evaluators participated. Slightly 
more than V^ (53%) report a tenure of 6 or more years, and over V\ (27%) have 
been involved for 4 to 6 years. Of the remaining evaluators, 10% have 1 to 3 
years' experience, while another 10% have been involved in PLA less than a year. 

Faculty who evaluate in each of the content areas available for PLA at 
NLU tcok part in this study; many of those evaluators perform assessments in 
multiple areas. In a related query, directors identify similar PLA content areas, 
although only of those available at NLU are offered for assessment at all 
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responding institutions. 

The size of PLA programs in terms of the number of evaluators involved 
during the past year ranges from a program with no more than 10 evaluators up 
to two programs employing more than 50 evaluators each. Appendix N illustrates 
the disparity in program size. The mode of the distribution is represented by the 
category of 11 to 20 evaluators, with 4 institutions employing a PLA faculty of this 
size. 

The size of PLA programs based upon the number of portfolios processed 
during the past year also varies greatly. One program reports a portfolio income 
of less than 50, and several an income of more than 500 ponfolios. Appendix O 
depicts the variation in program size. 

Other Descriptive Statistics 

In relation to evaluator credentialing and qualifications, directors and 
evaluators are in fairly close agreement as to the relative importance of 6 
performance characteristics. The most crucial characteristics for both groups is 
subject matter expenise, followed by personal objectivity. Other factors are rated 
quite closely by the two groups with the exception of philosophical orientation. A 
striking discrepancy is revealed here, with 64% of directors rating this 
characteristic as very important in contrast to the 37% of evaluators rating it this 
highly. The least imponant factor for directors and evaluators is motivation to 
maximize expertise in PLA with the majority of both groups rating this as only 
somewhat important to evaluator performance. Figure 6 portrays the 
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predominantly high degree of importance attached to the various characteristics by 
both groups of respondents. 

Figure 6 . Evaluator charactek 'sties rated as "very important." 

100-1 




Methods of ascertaining evaluator training needs practiced most widely by 
directors are very similar in frequency to those suggested by evaluators themselves. 
The method used by all directors and recommended by 83% of evaluators derives, 
in point of fact, from evaluator questions and suggestions. Both groups cite 
assessment staff observation as another important tool with an average response 
of 71%. Less salient but still of concern to both groups (35% average response) 
are student challenges or complaints as a form of diagnosis. Sixty-four percent of 
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directors view training needs assessment as an on-going process, and 55% conduct 
needs assessments on a regular basis, either formally or informally, A scant 27% 
of directors attempt to assess need only when some type of concern is expressed 
by an evaluator. 

Half of the directors surveyed hold mandatory orientation/training scissions 
for new evaluators compared with only 29% requiring training and development 
for experienced evaluators. Voluntary development offerings increase from 50% 
of the programs for new evaluators to 71% for experienced evaluators. At NLU, 
where faculty evaluator training and development has always been conducted on a 
voluntary basis, slightly over half of the respondents (53%) attended when they 
initially became active in PLA, and 63% repon not having attended an evaluator 
training session after their introduction to the assessment process. 

Directors and evaluators respond with similar perceptions in several areas 
pertaining to training needs for efficient functioning of new evaluators. For 
instance, all directors rate an understanding of PLA as an area of moderate to 
great need: 92% of evaluators concur. However, marked differences surface in 
others of the 10 areas identified. Whereas directors, for example, unanimously 
support compliance with suggested timeframes as a training need, only 68% of 
evaluators see this as a potential problem to be addressed. Other noticeable 
deviations appear in the degr^ie of need attached to relating learning outcomes to 
course equivalents, deter.nining overlap or duplication of credit, providing 
feedback to students, and using guidelines consistently and reliably. In all these 
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cases, evaluators rate the need for training significantly higher than directors. 
Overall, the least important training area to both groups is determining overlap or 
duplication of credit, with 64% of directors and 30% of evaluators identifying little 
to no need for training. Figure 7 illustrates perceived training needs for new 
evaluators. 

Figure 7 . Perceived training needs of new evaluators. 
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The same training areas are also considered as to degree of need for 
experienced e%'aluators. In this case, both groups rank all areas of need as 
significantly less than for new evaluators. 

Again, markedly different perceptions manifest themselves, the most 
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noticeable occurring in the area of compliance with timeframes as previously 
noted for new evaluators; 72% of directors find this an area of moderate to great 
need, but only 23% of evaluators agree. 

In a related question, 67% of evaluators indicate that they complete 
assessments within the requested two to three week turnaround time and 27% 
believe they average one week or less for evaluations. This has not necessarily 
been the experience of the Assessment Center staff, but again underlines a 
different viewpoint or perception- In other institutions, directors indicate that 
50% of evaluations are completed in one week or less and 30% take two to three 
weeks. This type of turnaround time may be due, at least in part, to the fact that 
full-time faculty evaluators are not located on multiple campuses, as is the case for 
NLU, 

Only about half as many evaluators as directors view providing student 
feedback as a prospective training topic. Conversely, evaluators find judging 
appropriate credit awards and determining overlap to be important more often 
than directors. Figure 8 illustrates perceived training needs for experienced 
evaluators. 
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Figure 8 . Perceived training needs of experienced evaluators. 




In addition to the 10 training areas proposed in the survey, directors and 
evaluators suggest a few related areas in which evaluators need training. Most 
suggestions pertain in particular to more experienced evaluators. 

Out of 9 possible professional growth opportunities and resources, 
workshop-type faculty development sessions are offered more widely than any 
other. Ninety-one percent of direaors have offered workshops at some time, but 
only 36% have done so during the past year. Nearly three-quarters of the 
directors have utilized regularly scheduled meetings and discussions of sample 
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essays, but again, few have done so within the past year (27% and 9% 
respectively). 

The most often used resource is informal staff conversations; not 
surprisingly, all directors offer this opportunity. In fact, this seems to be the 
primary method of faculty evaluator development Correspondingly, 100% of 
evaluators agree on the helpfulness of staff conversations on an informal basis. 
They also highly rate discussions of sample essays (97%) and workshops (90%). 

Two resources rated as helpful by evaluators, however, do not rate highly 
as director offerings. Bulletins and newsletters are viewed as helpful by 85% of 
evaluators, but are offered by only 36% of directors. An evaluator mentor is or 
would be helpful in the opinion of 81% of evaluators, while only 50% of directors 
have presented this option. Professional growth opponunities offered by directors 
and requested by evaluators are detailed in Figure 9. 
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Fi gure 9 . Professional growth opportunities. 
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The issue of compensation for evaluators seems to be a matter of some 
contention at NLU. At the time of this survey (August 1990), 86% of evaluators 
feh that compensation was unsatisfactory (the pay rate has subsequently been 
increased). NLU's flat fee per evaluation is similar to the compensation structure 
at 64% of the other institutions surveyed, although specific rates are not discussed 
However, 18% of the institutions offer an hourly evaluator rate and an additional 
18% provide some other means of compensation. 

This study examines a number of quality assurance issues, including the 
availability of seven key PLA-related documents for evaluators. Among directors, 
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availability of a given document ranges from 67% to 82%, indicating a lack of 
certain basic assessment program elements. At NLU, in spite of the availability of 
each of the documents in some form, a stanling number of evaluators is 
unfamiliar with one or more of the documents. Whereas the majority (87%) is 
acquainted with a description of evaluator roles and responsibilities, for example, 
only about half express familiarity with the PLA faculty/staff handbook and a 
description of the overall assessment process. More strikingly, a mere 7% of 
evaluators are aware of the institutional rationale for crediting prior learning. Of 
some interest is the fact that for the remaining three documents (written 
guidelines on college-ievel learning equivalence, acceptable documentation of 
learning, and evidence of learning), 2/3 of evaluators express familiarity despite 
the assessment director's perceived weakness in these areas. 

Also impacting the issue of quality assurance is the type of evaluator 
feedback received by students. Directors at the institutions polled indicate that 
brief written comments (1-5 sentences) are used 82% of the time. Other means 
of feedback are fairly evenly distributed with the exception of personal student 
contact, which is involved in only 27% of the institutions. 

At NLU, a corresponding 83% of evaluators typically provide feedback 
through brief written comments, although over half the evaluators at times use a 
longer narrative evaluation. No personal student contact is initiated by NLU 
evaluators. Appendix P details the various means of evaluator feedback. 

Another important provision for quality assurance is a periodic check of 
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credit recommendations made by evaluators. Among directors surveyed this is 
most often addressed via routine review by assessment staff. Faculty advisory 
committees also perform this function in 27% of the schools. 

NLU evaluators concur that assessment staff review is necessary, also citing 
advisory committee review as next in importance. A third of the evaluators, 
however, would prefer involvement by colleagues in PLA, and the majority feel 
that review should be initiated by student request rather than on a routine basis. 

Respondents agreed resoundingly that assessment staff should be involved 
in the formation of assessment policy and procedures. Thereafter, opinions 
diverge. While 73% of evaluators express a desire to contribute to this process, 
less than half the directors utilize evaluator input in this respecL And whereas 
55% of directors involve division deans, only 13% of evaluators find this 
appropriate. Specific recommendations are detailed in Appendix Q. 

When asked to consider the imp'^rtance of various factors to evaluator 
performance, a diversity of viewpoints again becomes apparent. Contact with 
students is least valued overall, with 28% of directors and 41% of evaluators 
claiming it not important. The only other factor cited by any directors (18%) as 
unimportant to performance is appropriate compensation; 10% of evaluators 
agree. Directors assign some degree of importance to the other six factors, but a 
proportion of evaluators, in each case, finds them irrelevant. 

A rather conspicuous divergence appears in the matter of peer (evaluator) 
support. Ail directors deem this factor somewhat to very important, as opposed 



ERLC 



105 



to 71% of evaluators. The lack of urgency attached to peer support is further 
evidenced by the fact that, while 49% of evaluators have consulted with perhaps 
one or two other evaluators over the past year, more than 40% have not consulted 
with any other evaluators . Likewise, all directors place importance on availability 
of support services, whereas 75% of evaluators rate support as somewhat to very 
important 

Among the two groups, agreement is high on the issue of training and 
development opponunities, with 100% of directors and 93% of evaluators rating 
this factor as important to evaluator performance. Unfortunately, evaluators do 
not always act upon this conviction. As previously discussed, only about three- 
fifths have attended any type of faculty evaluator development. Perhaps this 
reluctance to become involved in continuing education activities stems from the 
fact that 93% of evaluators feel moderately to very confident in assessing prior 
learning. In light of the statistics on participation at NLU, the 18% of directors 
who have experienced some resistance to development efforts can probably be 
viewed as a very low ratio. 

Directors and evaluators also agree strongly, in principle at least, on the 
importance of satisfaction with assessment program procedures. Another area of 
accord is the need for satisfaction with or confidence in the academic validity of 
the PLA concept itself. Similarities and differences in perceived evaluator 
performance factors appear in Figure 10. 
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Fi gure 10 . Evaluator performance factors. 
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Interpretations 

Discussion of Finding s 

The following section details the research iindings gleaned from the 
individual data sources utilized in this study. The open-ended inter\aew, expert 
program evaluation report and questionnaire results are abstracted and discussed 
relative to the study's stated purpose. 

The interview with Cynthia Scarlett, who, at the time was Center for 
Educational Systems and Services Director, was conducted in August, 1990. Ms. 
Scarlett received the procedure for faculty evaluator training at new contract 
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colleges; this procedure is similar to the one employed during the time she 
trained NLU faculty evaluators. She gave her impressions of the most critical 
elements in training and made observations on National's program. 

At contract schools initiating a PLA program, formal recruitment of faculty 
evaluators is usually nonexistent. More commonly, it is done on an individual 
basis, perhaps with program administrators hand-picking departmental 
representatives. Building a small core group of evaluators dedicated to the PLA 
concept is preferable to an attempt to force it upon traditionally-ensconced faculty 
resistant to change. 

Each contract school is responsible for developing iis own institutional 
philosophy and program standards in conformity with GAEL guidelines and NLU 
examples. Each must reflect its own individual concerns and goals. 

An initial faculty training session of two to three hours in length is 
conducted for new evaluators. The background and philosophy of PLA are 
stressed since philosophical agreement, in coalition with content expertise, is 
essential to evaluator performance. Faculty expectations are addressed in terms 
of their assumptions about what assures quality in traditional and nontraditional 
education. A common faculty tendency is to require more from prior learning; 
this tendency must be recognized and dispelled. While consistency and reliability 
are valid concerns, PLA should not be held accountable to invalid concepts of 
what traditiona l education accomplishes. 

Evaluator training includes discussion about "learning" — its definition, levels 
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and types, along with different ways adults learn and how mat learning can be 
assessed. Evaluators are asked to think about these issues in terms of their own 
adult learning experiences. Course materials which address these concerns are 
reviewed and discussed as well, along with a faculty evaluator handbook and other 
pertinent program materials and requirements. The necessity of combining and 
balancing learning or theory with experience is emphasized. 

The impact of involvement in nontraditional education upon traditional 
teaching methods is also examined. Faculty generally become invigorated through 
working with adults, panicularly when they have the opponunity to teach some of 
the course woik. This positive effect, along with any concerns of the faculty, are 
imponant points for funher dialog. 

A great deal of the follow-up faculty evaluator development is done on a 
one-to-one basis, addressing specific questions and apprehensions. Evaluators are 
usually sent the same sample essay to read, but not formally assess. Then the 
evaluators reconvene to discuss the essay as a group and make a credit 
recommendation. This allows them to perform their first assessment in a 
nonthreatening, low-pressure mode. It might be necessary at times to conduct this 
kind of exercise on a smaller scale by sending an essay and discussing it via phone 
conversation when training sessions or personal meetings are not plausible. 

Several problematical conditions at NLU have a wider application to PLA 
in general. One such condition is a concern with recognition from upper 
administration of the vital role of PLA within the institution. Assessment must be 
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an institutional priority in order to maintain program quality and academic 
viability. By communicating this support, administration can provide appropriate 
incentive for effective participation. Whether encouraged by means of monetary 
compensation or other "perks," faculty members need to view assessment as an 
integral part of their university duties. 

A second area for consideration is student-evaluator interaction. Contact 
between the two primary parties involved in assessment is a natural extension of 
the potentially dynamic nature of the process. In the usual assessment format at 
NLU, there is no student-evaluator contact unless a student requests clarification 
of an evaluation. 

Finally, communication among evaluators is limited. At the very minimum, 
a yearly evaluate/ meeting would allow an opportunity for discussion and 
refinement of PLA skills. Evaluators also need to be encouraged to work and 
consult together and with assessment staff throughout the academic year. 

A qualitative program review was conducted by Urban Whitaker of CAEL 
in December of 1989. Methodology was similar to the approach generally used 
for an accreditation visit and involved a preparatory review of materials succeeded 
by a site visit. At that time portfolios were examined, and administrators, 
students, evaluators, and assessment counselors were interviewed. 
Recommendations and a detailed summary of observations and suggestions were 
generated by Dr. Whitaker in keeping with appropriate standards of quality set by 
CAEL 
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A number of positive conclusions were reached; "on the negative side, 
virtually all deficiencies appear to be the result of inadequate professional 
development opportunities" (U. Whitaker, personal communication, 
December 11, 1989). This inadequacy is in direct conflict with GAEL 
Standard IX: "All personnel involved in assessment of learning should receive 
adequate training for the functions they perform, and there should be provision 
for their continued professional development" (Whitaker, 1989, p. 79). Hence "the 
primary recommendation resulting from this review is that . . . NLU should 
significantly expand the professional development opportunities available for 
assessors and evaluators of prior learning" (U. Whitaker, personal communication, 
December 11, 1989). 

Some specific concerns identify the need for a more developed Faculty 
Evaluator Handbook, more sufficient financial reward for evaluators, more specific 
and pointed recommendations to students and clearer delineation of roles among 
instructors, assessment counselors, and evaluators. Interestingly, another personal 
bias in favor of face-to-face contact between evaluators and petitioners was 
expressed Since quality assurance is contingent on professional development 
more than on any other single element, several suggestions were made for 
implementing expanded professional development programming: 

1) Scheduling meetings among evaluators to read essays and compare 
conclusions and between evaluators and other assessment personnel. 

2) Sending more evaluators and assessment personnel to professional 
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meetings on PLA. 

3) Offering workshops for evaluators, portfolio course instructors and 
assessment staff. 

The emphasis placed on increasing professional development activities is 

striking, surfacing more than once in Dr. Whitaker's report Many of the 

questions raised during the site visit stem from a deficiency in this area. 

Ultimately, the only hope of resolution lies in, but should not be limited to, 

regular meetings of evaluators with each other and periodic 
"retreats" at which those in all of the imponant roles (including 
some students and alumni) can exchange viewpoints, challenge each 
other's assumptions and practices and collaborate on a lot of 
important things that none of us ever has ample time to consider 
while we are "at work" (U. Whitaker, personal communication, 
December 11, 1989). 

The fact that at least 60% of NLU's evaluators are not full-time faculty 
members has certain implications for professional development activities. 
Logistically, it is difficult to convene the evaluators as a group for development 
purposes. Furthermore, evaluators may not readily have the opportunity to 
interact on an informal basis, particularly since even those full-time faculty 
members are based at more than one campus location. Finally, an assumption of 
shared institutional philosophy and rationale could be erroneous in this situation. 
While a mix of participants in the evaluator pool seems to be the norm in PLA, 
that fact probably complicates training and development efforts in several 
respects. 

Questions pertaining to recruitment methods and degree of evaluator 
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activity provide no particular insights. They are, however, of p>ersonal interest to 
the researcher as a basis for comparison. Likewise, PLA content areas are not 
especially pertinent to the study, except as a matter of personal interest in order 
to learn about other programs. 

Since 80% of the responding evaluators have been involved in PLA for 
four or more years, their interests lie with continuing education and development 
rather than training in new knowledge and skills. Laird (1985) thus distinguishes a 
need for specialized orientation in goals, policies and procedures, expansion of 
existing skills and growth maintenance. A complemental consideration is the 
possibility of nontraining solutions for performance problems not caused by a lack 
of knowledge or skills (Laird* 1985). The major thrust of NLU evaluator training 
will clearly be for those more experienced; this fact heightens the need for an 
effective and extensive program of initial orientation for the minority of new 
evaluators. It further underlines the importance of assessment staff participation 
in ongoing, active recruitment of evaluators in order to introduce "new blood" into 
PLA, thus revitalizing and invigorating the process. 

The study reveals no clear-cut correlation between amount of training 
offered and program size as measured by either evaluators involved or portfolios 
processed. Although the two largest (and most well-established) programs 
surveyed offer 6 out of 9 of the professional development opportunities cited, this 
figure is exceeded by the smallest program participating. Surprisingly, despite the 
fact that the contract schools have all benefited from very similar, if not identical. 
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initial training, subsequent efforts have departed somewhat from the NLU model. 
Since they are relatively young programs, however, the issue of faculty evaluator 
training and development presumably has not yet been addressed extensively, 

Evaluators and directors are generally in agreement on the importance of 
various performance characteristics. A discrepancy in perceived imponance of 
philosophical orientation, however, infers that NLU evaluators may not be fully 
cognizant of the implications for PLA of their frames of reference and personal 
biases. Philosophical orientation, that is. alignment with the basic tenets and 
standards of PLA, is completely fundamental to successful functioning of the unit, 
yet cannot be taken for granted. 

Considering the critical nature of professional development adamantly 
reinforced by PLA experts, it is disturbing to note that only half the directors 
surveyed hold mandatory training for new evaluators and less than a third require 
it for experienced evaluators. Even voluntary training is not offered by every 
institution on a regular basis. This is a serious concern for quality assurance and 
retlects on the validity of evaluations taking place. Correspondingly, the number 
of NLU evaluators who have not attended development activities is 
problematically high. Expanded offerings must be provided and participation 
increased in order to maximize expertise and promote excellence in the tleld. 

Based on the results of this study, directors need to place more importance 
on the training needs of new evaluators. Evaluators themselves express more 
concern with various areas of skill development than is exhibited by directors. 
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This may contribute to a substantial amount of discomfort or confusion on the 
part of evaluators and can be remedied early on through more extensive training 
which addresses a broad range of issues, 

Evaluators, on the other hand, appear relatively unconcerned with 
timeframes, while directors experience problems with compliance in this area. 
Certainly a lack of regard for requested turnaround time has created tensions at 
NLU, Evaluators, both new and experienced, obviously need to be made aware of 
the importance of timely assessments. 

Other training areas should be re-examined and re-evaluated from 
reciprocal viewpoints; issues are not necessarily weighted equally by evaluators 
and the directors responsible for their training. Each party to the PLA process is 
accountable to identify and address concerns for the mutual benefit of colleagues, 
peers, and ultimately, their students. 

As might be expected, the majority of evaluator training and development 
is accomplished on an informal, one-to-one basis: evaluators and directors agree 
on the effectiveness of this method. They also agree that discussions of sample 
essays and evaluator workshops are important training techniques. Directors, 
however, do not offer more creative opportunities often enough; newsletters and 
evaluator mentors, for instance, ahhough requiring more time and effort to 
coordinate, are highly viewed by faculty, A nontraditional approach to education 
such as PLA invokes training methods less traditional than the ubiquitous 
workshop. 
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Qearly quality assurance warrants improvement in a number of areas. 
Availability of and familiarity with PLA documents and assessment aids are two 
such areas. The quality of feedback to students is difficult to measure except 
subjectively, but monitoring is crucial to program success. Related to student 
feedback is the fact that, in spite of the preference of previously discussed 
evaluator trainers, personal student contact is rarely initiated in the evaluation 
process. Factors such as time, distance, expense, and perceived pressures inhibit 
this practice and perhaps reduce the potentially dynamic interplay in PLA- On 
the other hand, it can be argued that a greater degree of objectivity is maintained. 

In general, evaluators express a desire to be more involved in the entire 
PLA process, from formation of assessment policy and procedures to participation 
in advisory committees and collegial review of credit recommendations. If 
evaluators truly wish to increase their involvement, however, they need to 
acknowledge the importance of these functions on a routine, continuous basis, not 
merely when a complaint or crisis arises. Directors, if this is the case, can then 
more fully involve evaluators as valuable resources and panners in PLA, enlisting 
their input and mutual support. Increased participation will demand dedication, 
conviction, and an openness to new learning and professional self-improvement. 

Accomplishment of Purpose 

The primary purpose of this study is to assess training and development 
needs of evaluators. thereby validating the hypothesis that NLU*s assessment 
program will be strengthened by more comprehensive continuing professional 
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education. This objective has been accomplished, with research proceeding as 
planned. It is abundantly clear from the literature review and other sources of 
expertise that, as expected, professional development is the lifeblood of any PLA 
program. While NLU's program does not compare unfavorably with those of 
other institutions, evaluator training does not receive the attention it should from 
PLA directors and evaluators alike. The research questions have illuminated 
areas of interest and relevance to training and development and pinpointed NLU- 
specific concerns. Direct correlations are difficult to establish between, for 
example, length of tenure as an evaluator and perceived training needs. However, 
the more global purpose of verifying and substantiating the need for program 
development in faculty evaluator training and continuing education has been 
achieved. 

Recommendations 

Dealing with the Problem 

Accomplishments to date . Even prior to the completion of this study, the 
nature of the research precipitated developments in faculty evaluator training and 
nurtured positive results in the effort to maintain excellence in NLU's assessment 
program. 

A preliminary analysis of some of the questionnaire items served as the 
impetus for a workshop presentation at the Center for Educational Systems and 
Services Annual Degree Completion Program Conference in April 1991. Charts 
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were compOed illustrating training needs of both new and experienced evaluators, 
evaluator performance factors, and training resources. Based upon those 
responses, a model for an all-day evaluator training workshop was developed and 
presented. It included rationale, a detailed planning timeline, an agenda of 
activities, a list of materials, sample correspondence and a workshop evaluation. 
The model was fully developed so that it could be personalized and immediately 
implemented; it appears in Appendix R. Reaction from contract school PLA 
staff members in attendance was positive, indicating the value of a working model 
for planning purposes. 

In May 1991, a series of evaluator informational meetings was planned and 
conducted at three campus locations. Small groups of evaluators were updated on 
program specifications. Philosophy and procedures of PLA were reviewed, and 
evaluators were given the opportunity to express and discuss areas of concern, 
questions and problems they have experienced. Attendance at these sessions was 
limited, probably due to the proximity of the meetings to the end of the spring 
term. Attendees, however, appreciated the opportunity to express their views and 
interact with peers and PLA staff. 

Over the past summer, a supplemental Evaluator Handbook was prepared 
in collaboration with fellow assessment staff members. Existing evaluator 
materials have been incorporated with new information to more clearly set forth 
PLA philosophy, rationale, policy, procedures and practice. Evaluator roles and 
responsibilities are outlined, along with assessment guidelines. The supplement. 
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which appears in Appendix S, is presented in conjunction with the Portfolio 
Handbook (Student Guide), a list of suggested subject areas for learning 
experience essays, and a series of essay topic outlines for a complete evaluator 
reference document Much of the information contained therein had not 
previously been available in a written format 

A half-day evaluator workshop was organized, with Assessment Center 
cooperation and support, and delivered in October 199L This workshop was 
condensed from the model developed for the April CESS Conference; 
participants included all Assessment Center staff members and 14 faculty 
evaluators. Although only about one-third of the evaluator pool attended this 
Saturday workshop, the program was well received. The workshop agenda 
appears in Appendix T. Consensus of evaluators present suggests a desire to 
participate in some type of training and development session a minimum of once 
or twice yearly. 

Future implementation . In view of the research findings and in addition to 
the preceding items prompted by those findings and already effected, the author 
of this thesis recommends the following six goals for implementation in relation to 
faculty evaluator training and development at NLU: 

1) Increase evaluator participation in training and development 
activities. 

2) Develop an apprenticeship system for new evaluators. 

3) Provide Assessment Center support for evaluators as self-directed 
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learners. 

4) Expand PLA training and development activities for all evaluators, 
both new and experienced* 

5) Institute a system of checks and balances to safeguard quality and 
integrity in PLA 

6) Upgrade the standing of assessment coxmselors to faculty status. 
A discussion of each individual recommendation follows. 

As previously argued* participation in continuing professional education is 
vital to faculty functioning, particularly in a nontraditional arena such as PLA 
While a small number of evaluators always attends any planned activity, a much 
larger contingent is highly desirable. Offering a greater number of activities at 
various times of the day, week, and year can make it possible for more evaluators 
to find a time to attend. Often the same evaluators consistently attend meetings; 
it is imperative that ail evaluators be made aware of the need to meet periodically 
with each other and with Assessment Center staff. Barriers to participation, both 
in training and in the PLA process in general, must be identified and overcome, 
whether they be lack of awareness, poor perception of the process, or insufficient 
integration of PLA with general academic activities. Incentives for participation 
must be provided, not only by the Assessment G^nter, but also by higher 
administration. Evaluators should be strongly encouraged to provide input on 
their training needs and then to participate in remediation and development 

New evaluators need to undergo a period of supervised training to orient 
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and prepare them for PLA and to ensure the development of appropriate skills 
and techniques. This type of training program could feasibly be planned as an 
apprenticeship system, or alternatively as a formal course or an independent 
learning project The sample guidelines for an evaluator apprentice system at 
Central Michigan University's Institute for Personal and Career Development 
(Knapp, 1977) involve the following steps: 

1) A period of passive observation of staff meetings and of 
familiarization with program policies, procedures, credit criteria, title 
taxonomy and application forms. 

2) A period of practice evaluation utilizing previously evaluated 
applications. 

3) A period of probationary evaluation; the new evaluator makes 
limited original evaluations and submits them to the coordinator, 
who comments and returns them to the new evaluator. 

4) Full autonomy as an evaluator upon recommendation of the 
coordinator. 

Other aspects of the university's system would not be applicable to the NLU 
situation, but the above guidelines are readily adaptable. Evaluators should attend 
a ponfolio presentation session to learn how students are assisted in identifying 
experiential learning outcomes and planning for their assessment Evaluators also 
need to attend introductory training sessions designed to familiarize them with 
their new roles and responsibilities. Interviewing experienced evaluators to 
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determine specific criteria they have found most relevant in their practice would 
be a useful activity at this point A 'iDreaking-in period" of performing practice 
assessments would relieve new evaluators of the pressure of immediately 
determining credit awards and would allow assessment staff the opportunity to 
provide feedback and guidance. New evaluators should perform actual 
assessments only when both the evaluator and the assessment staff feel confident 
that quality can be ensured. Ideally an apprenticeship program of this nature 
should be mandatory for all new evaluators. 

The third recommendation recognizes faculty evaluators as self-directed 
lifelong learners and seeks to motivate and support them in learning about PLA. 
Evaluators should have access to relevant literature in the fields of adult learning, 
nontraditional methodology, experiential learning and assessment alternatives. 
Other resource materials should include portfolio development course curricula, 
sample portfolios and essays and pertinent workshop outlines. As a stan toward 
that end, an Assessment Center Library is being formed. Articles and selections 
should also be made available at meetings and, periodically, via a PLA newsletter 
or bulletin. A system of support personnel should be arranged to include a 
learning resource contact (Assessment staff) and a network of evaluator mentors 
in both the PLA process and specific subject matter requirements. 

While evaluators should be encouraged to act independently as self- 
directed learners, the necessity of expanding formal and informal developmental 
activities on a more advanced level cannot be overiooked. Whether termed 
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workshops, seminars, conferences, or bramstorming sessions, creative and 
productive measures must be formulated and conducted to further promote 
understanding and communication among all those involved in PLA Some ideas 
for evaluator exercises to be incorporated include: 

1) Developing learning outcome statements and updating topic outlines 
in fields of expertise. 

2) Discussing and learning new assessment methods such as oral 
interviews and product assessments, Evaluators who have employed 
these techniques can make presentations and lead discussions. 

3) Assisting a student (or use student information) to develop personal 
learning outcomes in fields of expertise. 

4) Participating in assessment simulations and recommending 
procedural improvements. 

5) Learning about recording and transcripting procedures in terms of 
department and title taxonomy; critiquing and evaluating 
procedures for clarity and academic acceptability. 

6) Designing and/or implementing a training model or component for 
evaluator development. 

7) Identifying possible learning experience essay topics from sample 
student autobiographies. 

8) Familiarizing with the entire portfolio and recognizing how individual 
essays fit into that schematic. 
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To the best of this researcher's knowledge, the above exercises have not 
heretofore been employed in NLU evaluator training and provide a broader 
perspective on PLA and issues of more interest and concern to experienced 
evaluators. Probably the single most useful tool for new and experienced 
evaluators alike is the practice assessment and discussion of sample essays. 

In addition to the preventive measures already described, periodic 
reliability studies should be conducted by having multiple evaluators make 
recommendations on a learning experience essay. This will help to ensure fairness 
in evaluation and identify potential training needs. Routine screening of credit 
recommendations by the Assessment Center Director or other appointed staff will 
also provide a measure of quality assurance to the PLA process. Figure 11 
illustrates how disagreement over an evaluation outcome can lead to a proposed 
appeal procedure initiated by any one of three sources. Another appropriate 
quality assurance measure which should be conducted occasionally is an evaluation 
of the evaluators or the evaluation process by students, faculty, administration and 
assessment staff. Standardization and articulation among evaluators and between 
university departments should be aspired to insofar as possible. 
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Figure IL Essay appeal procedure. 
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Essay Evaluation 




Advisory Commrttae 
Arbitnuion 



Binding Dedslon 



Evaluatof Tralning/Davak>pnisnt 

Finally, an infrastructural reorganization granting nontenure track faculty 
status to assessment counselors is recommended This academic upgrade will 
facilitate the synchronization of evaluation activity through recognition of 
assessment staff as PLA process experts. It will lend credence to their position as 
trainers of faculty evaluators and recognize their credentials in the field. 
Funhermore, it will allow them to work more directly with students in identifying 
prior learning experiences, analyzing learning outcomes, and presenting petitions 
for academic credit. 
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Progress has been made in some areas of faculty evaluator training and 
development at NLU. Planning and implementation are ongoing and must be 
given sufficient priority, personnel, and resources to continue to enlarge upon 
what has already transpired. As noted in the 1991 NLU accreditation report by 
the Commission on Institutions of Higher Education of the North Central 
Association of Colleges and Schools, regular review of the PLA program and its 
inclusion in the developing university-wide quality assurance process are vital to 
the future of the institution. 

Further Research 

A great many avenues exist for further research in faculty evaluator training 
and development since it is a relatively unexplored field. More scholarly research 
should be encouraged to further this specific body of knowledge. Future degree 
candidates can be steered into prior learning assessment and evaluator training as 
viable topics for study. Personnel already involved in PLA as staff members or 
administrators can utilize their positions to conduct studies in specific aspects of 
the field. Faculty acting as evaluators can pursue assessment issues from their 
unique perspectives for possible publication and professional credentialing. 

Various facets of PLA evaluator training fall beyond the scope of the 
present study. A number of the previous recommendations are appropriate for 
future research, such as program development and evaluation in evaluator 
apprenticeship, self-directed learning resources, and continuing professional 
activities. The questions of quality assurance and student satisfaction are ripe for 



ERIC 



126 



116 

study. So, too, is the matter of barriers to faculty participation in training and 
development, as well as overall faculty altitude toward prior learning assessment 
A vast field of study remains to be dealt with by future researchers. 
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APPENDIX A 



NATIONAL-LOUIS UNXVERSITY 



Campuses, Academic Centers and Educational Contracts 




# CAMPUSES 
Evanston Campus 
Chicago Campus 
West Suburban Campus 



□ ACADEMIC CENTERS 
Atlanu, Georgia 
Tampa, Florida 
St Louis, Missouri 
McLean, Virginia 
Milwaukee, Wisconsin 



<> FUTURE ACADEMIC CENTERS 
Philadelphia, Pennsyh^nia 
Seattle, Washington 

INTERNATIONAL CENTERS 
Heidelberg, Germany 
Seoul, Korea 



> EDUCATIONAL CONTRACTS 

Alaska New York 

Hawaii Ohk) 

Illinois Oregon 

Indiana Pennsylvania 

Kansas South Dakou 

Mkhigan Tennessee 

Minncsou West Virginia 
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Course Sequence 

NATIONAL-LOUIS UNIVERSITY HELD EXPERIENCE PROGRAM 
Bachelor of Arts in Management 

Term I 

Introduction to the Program 
Dynamics of Group Behavior 

* Adult Development and Learning Assessment 
Introduction to Research Methodology 

Term II 

Organizational Analysis 

Managing Interpersonal Communication 

Independent Study: Applied Research Project 

Term III 

Organizational Communication 
Research: Data Analysis and Presentation 
Cultural Influences in the Work Place 
Independent Study: Applied Research Project 

Term IV 

Principles of Management and Supervision 
Values and Ethical Decision Making 
Integrating the Applied Behavioral Science 
Independent Study: Applied Research Project 

* Module in which portfolio is completed and submitted to the Assessment 
Center 

Emphasis is placed on an understanding of adult development and learning 
processes. Each student conducts a personal and professional assessment 
and documents this by developing a portfolio. Additional credit for prior, 
extracollegiate learning may be earned through this portfolio. 
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Course Se^ ence 

NATIONAL-LOUIS UNIVERSITY FIELD EXPERIENCE PROGRAM 

Bachelor of Arts in Management 
and Education of Allied Health 

Term I 

Dynamics of Group and Organizational Behavior 

* Career Assessment and Planning 
Systems Management 
Research Project 

Term II 

Effective Interpersonal Relationships 
Statistical Methods and Research 
Legal Issues and Law in Health Care 
Presentation Skills in Allied Health Education 

Term III 

Principles of Management and Supervision 
Financial Management 

Human Resource Development for Health Care Supervisors 
Current Issues in Health Care 
Research Project 

* Module in which ponfolio is completed and submiited to the Assessment 
Center 

Each student prepares a ponfolio of professional and personal life-learning 
experiences which is assessed by faculty members and may earn up to 60 
quarter hours credit. Career patterns are reviewed and analyzed and goals 
are assessed through class activities. Future trends are examined in 
relation to changing careers and lifestyles. 
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Evaluator Guidelines 

Learning acquired in situations outside the sponsorship of an organization 
are evaluated through written essays. Each learning experience essay begins with 
a petition for credit which includes the department, course title, amount of credit 
requested, and a brief description of the learning outcomes demonstrated in the 
essay. 

The students are required to discuss their learning following the processes 
outlined in the Kolb Model (see next page). In the essays students must: 

1. Discuss their direct experience(s). 

2. Make observations and reflections concerning the learning derived 
from these experiences. 

3. Form abstract concepts and generalizations about the underlying 
principles involved with these learning outcomes. 

4. Demonstrate some application of these principles to new situations. 
The learning experience essays are presented to faculty members who have 

expenise in the appropriate subject areas. In evaluating the essays, the faculty 
evaluators examine how well the student has been able to incorporate the four 
parts of Kolb's Model, looking for college equivalent knowledge that includes 
generalizations and concepts as well as the specific experiences from which their 
knowledge was gained, and the specific applications of that knowledge. Finally, 
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the evaluators consider the documentation presented, determining whether or not 
it appropriately demonstrated the knowledge resulting from these learning 
experiences. All essays must have documentation. As in the evaluation of 
traditional college courses, the evaluator may use slightly differing individual 
standards in assessing credit, although all follow the general guidelines outlined 
above. 

KOLB'S MODEL OF EXPERIENTIAL LEARNING 
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ASSESSMENT CENTER EVALUATOR INFORMATION SHEET 
Name: Soc Sec #: 



Work Address: Work Phone:{ )_ 



Home Address: Home Phone:( 1. 

Please list the number of years of college teaching experience, 

with National College: with other insiituiions: 



Please list educational background. (Degrees received with Major/Minor areas of concentration). 



Please give a professional reference who may be contaaed. 

Name:_ Phone:{ )_ 

Address: 



Please list areas you are qualified to evaluate. 



Pertinent information or comments: 
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Cover Letter for Questionnaire 
Director 



July 23, 1990 



Otis O. Lawrence, Ph.D. 
Director, Assessment and 

BOG/BA Degree Program 
Governors State University 
University Park, IL 60466 

Dear Dr. Lawrence: 

Thank you for agreeing, at the GAEL Regional Meeting in April, to assist me in 
the research for my master's thesis on faculty evaluator training and development. 
Ultimately my goal is to identify and address specific training needs in order to 
promote the highest quality in the assessment of prior learning. 

Your expertise and cooperation in responding to the enclosed survey are vital to 
my study. The questionnaire should take approximately 20 minuter to complete; 
please return it to me in the enclosed postage-prepaid envelope BEFORE 
AUGUST 20, 1990. Copies of any of your pertinent training documents would also 
be most helpful. 

If you are interested in the results of this study, 1 would be glad to send you a 
summary. Please feel free to call me, should you have any questions, between 
9:00 AM and 5:00 PM CST at (708) 691-9390, ext. 2320. 

I truly appreciate your consideration in this matter. 

Sincerely, 



Lenice C Abbott 

Senior Assessment Counselor 

Enclosure 
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Cover Letter for Questionnaire 
Evaluator 

July 23, 1990 



Dear Evaluator: 

I need your assistance in the research for my Master's thesis on faculty evaluator 
training and development. My primary goal is to identify and address specific 
training needs in order to promote the highest quality in the assessment of prior 
learning at National-Louis University. 

Your cooperation in responding to the enclosed survey is vital to my study. The 
questionnaire should take approximately 15 minutes to complete; please return it 
to me in the enclosed postage-prepaid envelope BEFORE AUGUST 20, 1990. 

The Assessment Center will utilize your comments and suggestions in planning 
evaluator offerings for this academic year, which I hope you will look forward to 
attending. Please feel free to call me, should you have any questions, between 
9:00 AM and 5:00 PM at (708) 691-9390, ext 2320. 

Thank you for your participation. 

Sincerely, 



Lenice C Abbott 

Senior Assessment Counselor 

Enclosure 
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APPENDIX H 



EVALUATOR TRAINING SURVEY (D) 



Name: Date: 



Title: 

Institution: 

Complete this survey from the perspective of your institition's Prior Learning 
Assessment (PLA) program. Evaluators (assessors) are defined as the persons 
who judge prior learning experience essays to determine levels of learning and 
assign appropriate academic credits. 

For each question, circle the letter(s) or number(s) corresponding to your 
response; circle all that apply. 

PLEASE MAIL SURVEY BY AUGUST 20. 1990 . 

1. How frequently is each of the following groups involved in evaluating or 
assessing students' prior learning? 

Always Frequently Seldom Never 



a. Assessment program staff 1 2 3 4 

b. Individual faculty members 1 2 3 4 

c. Faculty committees 1 2 3 4 

d. Outside experts 1 2 3 4 

e. Others (specify) 1 2 3 4 



Z How are these evaluators selected? 

a. Recommended by department heads 

b. Recommended by other evaluators 

c. General invitation to faculty 

d. Recruited by assessment staff 

e. Appointed by administration 

f. All faculty MUST participate in PLA 

g. Other 
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3. Rate the importance of each of the following characteristics in overall 
evaluator performance. 



a. Philosophical orientation 

b. Subjea matter expertise 

c. Academic credentials in subject 

matter 

d. Knowledge of assessment process 

e. Objectivity 

f. Motivation to maximize expertise 

in PLA 



Very Somewhat Not 

Important Imponant Important 



1 2 3 

1 2 3 

1 2 3 

1 2 3 

1 2 3 

1 2 3 



4. How do you ascertain evaluator training needs? 

a. Written surveys 

b. Evaluators' questions, suggestions 

c. Assessment staff observations 

d. Student complaints 

e. Expert outside sources 

f. Other 



5. When do you conduct, either formally or informally, training needs 
assessment? 

a. Regularly (how often?)_ 

b. On-going 

c. Other (specify) 

d. Have not addressed this issue 
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6. Please indicate which of the following documents are available to evaluators 
for your program. 

Available Not Available 

a. Institutional rationale for crediting 

prior learning 1 2 

b. Description of the roles and responsibilities 

of evaluators 1 2 
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c. Faculty/staff handbook describing PLA policy, 



practice and procedures 1 2 

d. Detailed description of your overall 

assessment process 1 2 

e. Guidelines as to what constitutes college- 

level learning 1 2 

t Guidelines for acceptable documentation 

of learning 1 2 

g. Guidelines for adequate evidence of learning 1 2 



7. Are orientation/training sessions held: 

a. for new evaluators? 

1. mandatory attendance 2. voluntary attendance 

b. for experienced evaluators? 

1. mandatory attendance 2. voluntary attendance 



8. Please consider the degree to which you feel NEW EVALUATORS need 
training in each area below to function most efficiently. 

None Little Moderate Great 



a. 


Identifying documentation & 












substantiating learning 


1 


2 


3 


4 


b. 


Understanding rationale for 












crediting prior learning 


1 


2 


3 


4 


c. 


Deciding what constitutes 












college-level learning 


1 


2 


3 


4 


d. 


Relating learning outcomes to 












course equivalents 


1 


2 


3 


4 


e. 


Judging appropriate amount of 












credit to be awarded 


1 


2 


3 


4 


f. 


Providing, feedback to students 


1 


2 


3 


4 


g- 


Determining overlap or duplication 












of credit 


1 


2 


3 


4 


h. 


Using assessment guidelines 












consistently & reliably 


1 


2 


3 


4 


i. 


Complying with suggested timeframes 












for evaluation 


1 


2 


3 


4 


j- 


Delineating learning outcomes 


1 


2 


3 


4 
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9. To what degree do you feel EXPERIENCED EVALUATORS need ongoing 
training in the same areas? 

None Little Moderate Great 



f. 



Identifying documentation & 

substantiating learning 
Understanding rationale for 

crediting prior learning 
Deciding what constitutes 

college-level learning 
Relating learning outcomes to 

course equivalents 
Judging appropriate amount of 

credit to be awarded 
Providing feedback to students 
Determining overlap or duplication 

of credit 
Using assessment guidelines 

consistently & reliably 
Complying with suggested timeframes 

for evaluation 
Delineating learning outcomes 



2 
2 
2 
2 
2 

0 



2 
2 



3 

3 

3 

3 

3 
3 

3 

3 

3 
3 



4 

4 

4 

4 

4 
4 

4 

4 

4 
4 



10. In what other areas do you find evaluators need training and/or development? 
(Use reverse side if necessary) 

a. J 

b. ' 



c. 



d. 

e. 

f. none 
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Which of the following growth opportunity/resources have you offered to help 
your evaluators learn about PLA? 



a, Evaluator mentor 

b. Worshops/faculty development 

sessions 

(X Regularly scheduled meetings 
d Reading and discussion of 
sample essays 

e. Audio visuals on PLA 

f. Bulletins, updates, newsletters 

g. Retreats 

h. Professional conferences 

(GAEL, etc.) 

i. Informal staff conversations 

j. Other (specify) 



Have Offered in Have Not 

offered past year offered 

1 2 3 

1 2 3 

1 2 3 

12 3 

1 2 3 

1 2 3 

1 2 3 

1 2 3 

1 2 3 



12. Have you met with any resistance to faculty development attempts? If so, 
please describe briefly. 

a. yes 

b. no 

13. How important do you consider each of the following factors to evaluaior 
performance? 

Very Somewhat Not 

Important Important Important 

a. Training and development 

opportunities 

b. Appropriate compensation 

c. Student contact 

d. Assessment depanment staff relations 

e. Satisfaction with program procedures 

f. Satisfaction with academic validity 

of PLA concept 

g. Availablity of support services 

h. Mutual support of peers (evaluators) 



2 
2 
2 
2 
2 

2 
2 
2 



3 
3 
3 
3 
3 

3 
3 
3 
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14. Do evaluators sign a contractual agreement regarding assessment guidelines 
and expectations? 

a. yes 

b. no 



15. On the average, how many assessments does each evaluator perform in a 
year? 

a. less than 10 

b. 11-50 

c. 5M00 

d. 10M50 

e. 151-200 

f. more than 200 



16. On the average, approximately how much time is spent per essay evaluation? 

a. less than 15 minutes 

b. 15-30 minutes 

c. 30-60 minutes 

d. more than 1 hour 

e. other (specify) 

17. What is the average turnaround time for evaluations? 

a. 1 week or less 

b. 2-3 weeks 

c. 4-^ weeks 

d. 6-8 weeks 

e. over 2 months 



18. What kind of compensation, if any, is provided for your evaluators? 

a. flat fee per evaluation (please specify) 

b. hourly 

c. monthly 

d. other 

e. no additional compensation 
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9. What means of feedback do evaluators provide for students? 

a. credit recommendations only 

b. rank on rating scale 

c. brief written comments (1-5 sentences) 

d. narrative evaluation 

e. phone contact 

f. personal contaa 

g. other 

h. none 



-0. What provisions, if any, have been established for review of credit 
reommendation made by evaluators in terms of validity and reliability? 

a. by department chair 

b* by faculty advisory committee 

c. by division deans 

d* by other evaluators 

e* by assessment director/staff 

f* other (specify) 

g. none (skip to question 22) 

!• How is such review initiated? 

a. routine 

b. upon request of student 

c. upon request of assessment staff 

d. other (specifiy)^ 



1 Who, at your institution, is involved in assessment policy and procedures 
formation? 

a, evaluators 

b* assessment department staff 

c. traditional college faculty 

d. advisory committee 

e. department chairs 

f. division deans 

g. others ^ 
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23. How many evaluators have been involved in your program during the past 
year? 

a 1-10 

b. 11-20 

c. 21-30 

d. 31-40 

e. 41-50 

t over 50 



24. How many ponfolios have been processed through your program during the 
past year? 

a. less than 50 

b. 50-100 

c. 101-200 
d 201-300 

e. 301-400 

f. 401-500 

g. more than 500 



25. For each of the following content areas, please indicate whether or not PLA's 
are completed in your program. 

a. Allied Health 

b. Art 

c. Business 

d. Communication Arts 

e. Data Processing 

f. Education 

g. English 

h. Health Science 

i. Human Services 
j. Interdisciplinary 
k. Math 
1. Music 

m, Philosophy/Religion 

n. Physical Education 

o. Psychology 

p. Science 

q. Social Science 



yes 


no 


yes 


no 


yes 


no 


yes 


no 


yes 


no 


yes 


no 


yes 


no 


yes 


no 


yes 


no 


yes 


no 


yes 


no 


yes 


no 


yes 


no 


yes 


no 


yes 


no 


yes 


no 


yes 


no 
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Please feel free to contact me with any additional comments you might have on 
this topic. 



THANK YOU VERY MUCH FOR TAKING THE TIME TO COMPLETE 
THIS QUESTIONNAIRE. 
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APPENDIX I 



EVALUATOR TRAINING SURVEY (E) 



Date: 



Ibmplete this survey from your perspective as an evaluator in NLU's Prior 
naming Assessment (PLA) program. For each question, circle the letter(s) or 
lumber(s) corresponding to your response; circle ail that apply. 

^LEASE MAIL SURVEY BY AUGUST 20. 1990. 

1. Which best describes your position at NLU? 

a. Full-time faculty 

b. Adjunct faculty 

c. Outside expert 

1 How were you initially selected as an evaluator? 

a. Recommended by department head 

b. Recommended by another evaluator 

c. General invitation to faculty 

d. Recruited by assessment staff 

e. Appointed by administration 

f. Other 



1 Rate the importance of each of the following characteristics in your overall 
performance as an evaluator. 



Very Somewhat Not 

Important Important Important 



a. Philosophical orientation 
b* Subject matter expertise 
c. Academic credentials in subject 



1 
1 



2 
2 



3 
3 



matter 

d. Knowledge of assessment process 

e. Objectivity 

f. Motivation to maximize expertise 



1 
1 

1 



2 
2 
2 



3 
3 
3 



in PLA 
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4. How can evaluator training needs be ascertained? 

a^ Evaiuators' questions, suggestions 

b. Assessment staff observations 

c. Student challenges 

d. Expert outside sources 

e. Other 



5. How comfortable (confident) do you feel in assessing prior learning? 

a^ Very comfortable 

b. Moderately comfortable 

c. Not very comfortable 

d. Very uncomfortable 



6. Please indicate whether or not you are familiar with each of the following at 
NLU. ^ 



a/ Institutional rationale for crediting 
prior learning 

b. Description of the roles and responsibilities 

of evaluators 

c. Faculty/staff handbook describing PLA policy, 

practice and procedures 

d. Detailed description of the overall 

assessment process 

e. Guidelines as to what constitutes college- 

level learning 

f. Guidelines for acceptable documentation 

of learning 

g. Guidelines for adequate evidence of learning 



Familiar Not Familiar 

1 2 
2 
2 
2 
2 



1 



2 
2 



7. Did you attend an orientation/training session as a new evaluator? 
a yes 



b. no 



L mandatory attendance 
2. voluntary attendance 



ERLC 



152 



142 



8. Have you ever attended a training/development session for experienced 
evaluators? 

a. yes 

b. no 

9. Based upon your experience, please consider the degree to which you feel 
NEW EVALUATORS need training in each area below to function most 
efficiently. 

None Little Moderate Great 

a. Identifying documentation & 

substantiating learning 

b. Understanding rationale for 

crediting prior learning 

c. Deciding what constitutes 

college-level learning 

d. Relating learning outcomes to 

course equivalents 

e. Judging appropriate amount of 

credit to be awarded 

f. Providing feedback to students 

g. Determining overlap or duplication 

of credit 

h. Using assessment guidelines 

consistently & reliably 

i. Complying with suggested timeframes 

for evaluation 
j. Delineating learning outcomes 
k. Recognizing use of Kolb Model 
elements 

10. To what degree do you feel that you, as an EXPERIENCED EVALUATOR, 
have at some point needed training in the same areas? 

None Little Moderate Great 

a. Identifying documentation & 

substantiating learning 12 3 4 

b. Understanding rationale for 

crediting prior learning 12 3 4 



2 3 4 

2 3 4 

2 3 4 

.23 4 

[23 4 

12 3 4 

12 3 4 

12 3 4 

12 3 4 

12 3 4 

12 3 4 
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Deciding what constitutes 

college-level learning 
Relating learning outcomes to 

course equivalents 
Judging appropriate amount of 

credit to be awarded 
Providing feedback to students 
Determining overlap or duplication 

of credit 
Using assessment guidelines 

consistently & reliably 
Complying with suggested timeframes 

for evaluation 
Delineating learning outcomes 
Recognizing use of Kolb Model 
elements 



2 

2 

2 
2 

2 

2 

2 
2 



3 

3 

3 
3 

3 

3 

3 
3 



4 

4 

4 
4 

4 

4 

4 
4 



what other areas would you like to receive ongoing training and/or 
velopment? (Use reverse side if necessary) 



none 



r each of the following growth opportunities/learning resources, indicate 
lich you HAVE FOUND HELPFUL in the past or WOULD FIND 
5LPFUL if made available. 



Have found Would find 
helpful helpful 



Evaluator mentor 
Worshops/faculty development 

sessions 
Regularly scheduled meetings 



2 
2 



Not 
Helpful 



3 
3 
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d Reading and discussion of 

sample essays 
e. Audio visuals on PLA 
f* Bulletins, updates, newsletters 
g. Retreats 

Professional conferences 
(GAEL, etc.) 
L Informal staff conversations 
j. Other (specify) 
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1 2 3 

1 2 3 

1 2 3 

1 2 3 

1 2 3 

1 2 3 



13. How important do you consider each of the following factors to your 
performance as an evaluator? 

Very Somewhat Not 

Important Important Important 

a. Training and development 

opportunities 

b. Appropriate compensation 

c. Student contact 

d. Assessment department staff relations 
e* Satisfaction with program procedures 

f. Satisfaction with academic validity 

of PLA concept 

g. Availablity of support services 

h. Mutual support of peers (evaluators) 



2 
2 
2 
2 
2 

2 
2 



3 
3 
3 
3 
3 

3 
3 



14. With how many other evaluators have you consulted regarding PLA in the 
past year? 

a. none 

b. 1 or 2 

c. 3 or 4 

d. 5 or more 
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15- On the average, how many assessments do you perform in a year? 

a less than 10 

b, 11-50 

c. 51-100 
d 10M50 

e. 151-200 

f. more than 200 



16, On the average, approximately how much time do you spend on each 
evaluation? 

a. less than 15 minutes 

b. 15-30 minutes 

c. 30-60 minutes 

d. more than 1 hour 

e. other (specify) 



17. What is your average turnaround time for credit assessments? 

a. 1 week or less 

b. 2-3 weeks 

c. 4-6 weeks 

d. 6-8 weeks 

e. over 2 months 



18. Is the compensation you receive satisfactory? ($10/original submission, 
$5/re write) 

a. yes 

b. no 

your suggestion for improvement 

19. Which means of feedback do you typically provide for students? (circle all that 
apply) 

a. credit recommendations 

b. rank on rating scale 

c. brief written comments (1-5 sentences) 

d. narrative evaluation 
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phone contact 
personal contact 

other 

none " 

"hat provisions, if any, should be available for review of credit 
ommendations made by evaluators in terms of validity and reliability? 

by depanment chair 

by faculty advisory committee 

by division deans 

by other evaluators 

by assessment director/staff 

other (specify). 

none (skip to question 22) 

w should such review be initiated? 
routine 

upon request of student 

upon request of assessment staff 

other (specifiy) 



lo, in your opinion, should be involved in assessment policv and nroced 
mation? ' ^ 

evaluators 

assessment department staff 
traditional college faculty 
advisory committee 
department chairs 
division deans 
others 



w long have you been involved in the assessment of prior learning at NLU? 

less than 1 year 
1 - 3 years 
4-6 years 
more than 6 years 
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24. Please indicate in which content areas you complete evaluations. 



a. 


Allied Health 




Art 


c. 


Business 


d. 


Communication Arts 


e. 


Data Processing 


r 

L 


Education 


g- 


English 


h. 


Health Science 


I. 


Human Services 


j- 


Interdisciplinary 


k. 


Math 


1. 


Music 


m. 


Philosophy/Religion 


n. 


Physical Education 


0. 


Psychology 


P- 


Science 


q- 


Social Science 



COMMENTS: 



Please feel free to contact me with any additional comments you might have on 
this topic. 

THANK YOU VERY MUCH FOR TAKING THE TIME TO COMPLETE 
THIS QUESTIONNAIRE. 
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Schedule of Events by Months - 1990 



Activities 

Proposal Approved 

Final ize Population 

Refine Questionnaire 

Collect Data 

Interview CESS 
Director 

Review Expert 
Evaluation 

Analyze Data 



May 



June 
0 



MX 



Aug 



Sept 



Oct 



Nov 



Write Thesis Ch. 1 
Write Thesis Ch. 2 
Write Thesis Ch. 3 
Prepare Final Copy 
Submit Final Copy 



X 
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APPENDIX K 
Thesis Expense Budget 

Assessment 

llem Researcher' s Expense Center Fxp ensp. 

Word Processing $300.00 

Photocopying in Library 40.00 

Photocopying of Questionnaire $20.00 

Stationery and Supplies 9g qq 

Postage 3000 

Photocopying of Thesis 15 OQ 

Manuscripts 

Tliesis Binding Fee 30 qq 

'^^^a' $370.00 $85.00 
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Letter of Suppon 



May 23, 1990 



TO: Adult and Continuing Education Program 

FROM: Scott D. Heck 

Director, Assessment Center 

SUBJ: Letter of Suppon for Lenice C Abbott 



The ultimate success of any prior learning assessment program depends on valid 
and reliable assessment by expen evaluators. One of the key ways to guaranteee 
this quality is through continuing staff developmenL 

Lenice Abbott, Senior Assessment Counselor, has been given the responsibility to 
assist in the staff development process at National College of Education (NCE). 
Her research into the staff development processes and needs at NCE and other 
institutions is directly related to this activity. Therefore, she will be supported and 
encouraged in her effort by this office. 

This support includes the duplication, mailing and telephone services needed to 
complete her research. Also, the computer resources of the office will be 
available to her. While the writing of her thesis is considered to be outside of the 
job responsibilities, she will be able to work on many activities during normal 
working hours since much of the research is directly related to staff development. 

I see this research as beneficial to National College of Education, in particular, 
and to other institutions, in general, as quality assurance has the highest priority in 
prior learning assessment today. 
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APPENDIX M 

Assessment Performed by Evaluator 
in a One* Year Period 



% of Evaluators 


# of Assessments 


Evaluator Estimate 


Director Estimate 


Less than 10 


24 


10 


11-150 


28 


60 


51-100 


21 


30 


101-150 


10 




151-200 


3 ■ 




More than 200 


14 
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APPENDIX O 
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APPENDIX P 
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Personnel Who Should Be Involved 
in Assessment Policy and Procedures 
Formation 



Personnel 


% Evaluator Perception 


% Director Perception 


Assessment Dept. Staff 


90 


91 


Pvahiatnrs 






Traditional Faculty 


37 


27 


Advisory Committee 


43 


27 


Department Chairs 


27 


9 


Division Deans 


13 


5 


Other 


3 


18 



167 




APPENDIX R 



THE CTALCATCR FACTOR 



Prepared and Fresenred by 
Lenice Aljborr 
and 

Roberrz sell 



April 13. 1991 



Obiectives for tih.e Evaluatcr Facrsr 

To understand the perceived role of t.^*e evaluator from the perspectives 
of assessment center directors and evaluators themselves. 

To review a modal training proqran appropriate for new evaluators or as 
a refresner for those with nore experience. 

To generate ideas for adapting the sodel workshop to fit specific needs 
of your program. 



o 1 bo 
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MODEL EVALUATCR r7Ai:ii:c3 WORKSHOP 
Table ot Ccntenrs 



Inrroducrion to Workshop 3 

Woricshop Planning Timeline 4 

Daparwent Agenda 6 

Workshop Agenda 8 

Evaluator Workshop Motice ; 9 

Workshop confirmation 10 

Workshop Reminder XI 

Workshop Materials 12 

Evaluator Packets 13 

Workshop Evaluation 14 

Workshop Follow-up 15 
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MODEL EVALUATOR TRAINING WORKSHOP 



Introduction to WorKshop 



Almost evexryone seems to agree that evaluators of prior experiential 
learning need comprenensive and ongoing tramxng and development in 
order to fulfill tneir roles. However, identifying training needs, 
convincing evaluators of their relevance, and eliciting participation 
in worKshops or other activities are easier said tnan done. This 
Evaluators WorKshop has been designed to address these issues in 
programs that have previously lacKed formal training. While providing 
assessment overview for new or relatively inexperienced evaluators, 
more established evaluators will also nave the opportunity to review 
and renew practices # techniques and attitudes. As the basis for a one 
day worKshop, it can be personalized to fit program personnel, 
evaluator personalities, training needs^ time and cost considerations. 

A large scale evaluator workshop will be beneficial on at least a 
yearly basis, coinciding with revisions or updates in Portfolio and 
Evaluator Handbooxs. At that time, .copies of the new HandbooKs and an 
update highlighting na;)or changes can be disseminated. It is 
imperative that all Assessment center Staff reccers play an active part 
in planning and conducting zTib workshop • 

Througnout the academic year, additional activities and opportunities 
for professional development are important. These night include: 

1. informal small grcup meetings of interested evaluators 

2. access to a collection of reaamgs in pricr learning assessment 

3. opportunities to mentor or networK informally with other 
evaluators 

4. conversations with Assessment staff wnen prcclems or questicns 
arise 

5. evaluator newsletters 

6. attendance at professional conferences 

Evaluators initiating assessment without attenamg a formal training 
session need to undergo an abbreviated one-on-one training with an 
Assessment staff member. 

We hope this model will prove useful to each of you within the 
frameworX of your cwn program. We clan to present it for cur own 
faculty evaluators m the Fall, and we welcome ccr:ments and suggestions 
from those of you wno mignt experiment with it at your institutions. 
Good lucX! 
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WORKSHOP PLANNING Tiy^IlIE 

6 - Q weeKs oner 

Schaduie any new or potenciaily interested evaluators to visit a class 
group on the Portfolio Presentation and counsciing nights. Provide 
copies ot the current Portfolio and Evaiuaccr Handbooks for review 
before commencing visits, complete class visits prior to the Evaluator 
workshop is possible. 

S - weeks prior 

Schedule date and reserve facilities for vorksnop. Begin to assemble 
preliminary workshop materials: 

Evaluator Workshop .'fotice 
worxsnop Confirracicn 
Preliminary Aqencia 
Pre-addressed Stanpea Hinvelopes 
Make directional signs to ne posted on the day of the workshop. Assign 
specific worKshop duties/ roles to each Assctsssent Center statf r.ember. 

4 weeks prior 

Send Evaluator worKsncp Notices to all current and new evaluators. 
Confirmations include space for evaluators to suggest possible agenda 
topics and should be returnea to the Assessment Center no later than 
two weeks prior to the workshop. This givei the departr.ent an 
opportunity to review and incorporate the :.ieas into the finalized 
agenda* 

Contact two or tr.ree well establishea evai-atsrs to participate with 
the Departr.ent Cirecrcr ^Moderatorj ir. tr.e Panel ::iscussion entitled 
'*what 13 a Learning Experience Essay?" Prepare a list of subtopics 
related to the theme for t^.elr discussion ana elicit their ir.put. 

Beam selection of sample essays :n varices cepartr.ents for Croup 
Evaluation exercise . 

I week prior 

Reconfirm room and tine. 

call each evaluator wno did not respond zz t^.e mailing to confirm 
attenaance . Sena worjcsnop Reminaers zo zr.zse who aid respona, 
and compile fmalizea Participant List. 

Finalize workshop r.aterials. sample essays (Group Evaluation) and 
Agendas* Xerox neeaea copies and assemoie Evaluator Packets (see 
list) . 

Schedule meeting cf Assessment Director ar.d panel .T.embers to review 
their roles* 



4 



171 

BEST COPY AVAILABLE 



160 



5 dava prior 

MMt as a deparraenr to review worksnop acmviriea and confira roles of 
eacn sraff nemoer. Include room ser up^ resresnmenrs , luncn, nore 
taxing, clean up, ere. 

Prepare name tags for evaluacors (to mciuns major area of evaluation 
expertiso) and department neaoers. Prepare "buddy list:,'* pairing 
•valuators for Peer Evaluation exercise. 

1 dav prior 

Set up room configuration - conference style thar can Later be broken 
into small groups. Post directional sigr.s. 

1 hour cnor 

All department nembers arrive at r.eetmq sice. Set up tables for 
ref reshroentis and distribution ot Evaiuatcr Pacxets/ Name Tags. Set 
out/up any additional materials, AV equiprent, etc. 

Conduct wor)cshop 

Mote taker keeps minutes t.^rougnout tr.e workshop. This person also 
collects and compiles tr.e workshop evaluations. 

1 week after 

Edit and copy workshop minutes and send prcceedings to all participants 
vith Workshop Follow Up Letter. Sena proceedings and HTvaiuator PacKets 
'minus WorKsnop Evaiuaticni to tncse not attenaance. 

romoile evaluation results ana r.eet as a cepartr.ent to review. 
Incorporate results into r.ext training session. 



5 
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Deparrrenc Agenda 

8:30 - 9;00 a.m. 

coffee and Rolls 

Disrriburion of Evaluarcr Packers 



9:00 - 9:15 a.m. 

INTRODUCTIONS 
Assessaenr Cenrer Staff 
Evaluacors 
Ice Breaxer 



9: 15 - 9:30 a.m. 

WORKSHOP OBJECTIVES 

!♦ Outline the assessment process step by step. 

2. Identify guidelines ror essay evaluation. 

3. Highlight modifications in assessment polices & procedures 

4 . Examine nisconcepticns & l:.r:i*a*ions of tt:e assessment 
process. 

5. Increase communicaticn aaor.g evaluators, students & assessment 
staff. 



9:30 - 9:45 a.m. 
PHILOSOPHY 
the University 

council for Adult and Experienrizl -earning (GAEL) 
Assessment Center { programs servea) 



9:45 - 10:15 a.m. 

ASSESSMENT CENTER ROLE - OlSCVSSlZU 

Purpose 

Staff roles 

Responsiaility to students 
Responsicility to evaiuatcrs 
ResponsiDility to universi::y 
Appeals process - Advisory council 

10: 15 - 10:30 a.m, 
BREAK 



10:30 - ll:C0 a.m. 

EVALUATCR ROLE - OISCITSSICN 
Purpose - content experr 
Experiential learning noaels 
vaiue to program 
Responsibility to students 
Responsibility to Assessment sza.tt 
Responsibility to University 
Evaluatcr perceptions 
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11:00 • 12:00 noon 

PANEL DISCUSSION - WHAT IS A LEARNING EXPEHIENCE ESSAY? 

College level learning 

Credit fur learning . MOT experience 

Concepts vs. Experience 

Componencs (Experience, refleccion, concepcsr applicacicn) 

Oocximentation 

Format requirements 

overlap/duplication of credit 

Appropriate credit awards 

Evaluator feedback 

12:00 - 1:00 p. a. 
LUNCH 

1:00 - l:l5 p. a. 

MECHANICS or EVALUATICN 
Paperwork involve a 

Original sucraission, first ana secona rewrites 
Payment to evaluators 

l: 15 - 2: 15 p. a. 

PEER ET/ALUATIONS 

Announce '*3uddies" and J;ave evaluators break off m pairs. 
Eacn evaluator writes a 1 page r.ini-essay (rougft draft: on a topic 
of cnoice. Essay stiould neet tr.e general requirements as outlined 
in tne HandbooKS and include laaasi Z appropriate concepts and 
supporting experiences. Excnange essays with "Buddies'* and 
conduct small scale evaluations using ::)iscussion Guidelines. 

General ;:iscussion/Clcsure tc £:xercise 

2:15 - 2:20 p..T». 



2:30 - 3:30 p. a. 

GROUP ET/ALUATIOHS 

BreaK evaluators into cuzs crcups of 5 or 6, preferably by areas 
ot expertise. Eacn group cnooses a reporter. 3istri::utG sample 
essay, .-tatcning topics to grcuo expertise; use Discussion 
Guidelines from previous exercise. Eacn group r.ust receive a 
different essay. Groups reaa and individually evaluate essays, 
then discuss findings as a grcup and attempt to reacn agreement on 
final creoit award. 

General ::iscussion - group reporters sunrarize fcollect sample 
essays ) 

3:30 - 4 : OC p. a, 

WRAP UP /WORKSHOP EVALUATION 

Address* any topics from Worxsnop Confirmations not yet covered. 
General questions/answers to staff, evaluators. Distribute 
worKsnop Evaluations (collect at door; . 



BREAK 
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Worxsnop Agenda 



8:30 - 9:00 a.m. 

coffee and Roils 

Oiscrxbucion of Evaiuacor Packers 

9:00 - 9: 15 a.m. 

INTRODUCTIONS 

9: 15 - 9: 30 a.m. 

WORKSHOP OBJECTIVES 

9:30 ~ 9:45 a.m. 
PHILOSOPHY 

9:45 - 10: 15 a.m. 

ASSESSMENT CENTER ROLE 

10:15 - 10:30 a.m. 
BREAK 

10:30 - 11:00 a.n. 

EVALUATOR ROLE 

11:00 - 12:00 noon 

PANEL DISCUSSION - WAT IS A LEARNING EXPERIENCE ESSAV? 

12:00 - 1:00 p.m. 
LUICH 

1:00 - 1:15 cm. 

MECHAJaCS OF EVALUATICN 

1:15 - 2:15 p.m. 

PEER EVALUATIONS 

2:15 - 2:20 p.m. 
BREAK 

2:30 - 3:30 p.m. 

GROUP ETMLUATIONS 

: : 30 - 4 :00 D.m. 

wrap' VP/WORKSHOP HVALUATIC:? 
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EVALUATOR WORKSHOP NOTICZ 



(Date) 



Dear Evaiuator: 

As you know, thorough faculty evaiuator preparation is an essential 
element: to any successful assessment program. 

Therefore, on (DAY), the Assessment Center will conduct an Evaiuator 
Workshop. This session will taice place at (PLACE/TIME). Enclosed you 
will find a copy of the preliminary r.eetmg agenda. 

To supplencnt our activities. Z would like to solicit your ideas. 
Pleas* fe«l free to offer your suggestions on tne enclosea worksnop 
confiraaticn sheet. For your convenience I have included a stamped 
pre-addressed envelope. vour response should ce nailed no later than 
(DATE) . 

I know you'll find our day to ba quite exciting and I strongly urge you 
to attend. You'll not only have an opportunity to meet with your 
peers, but you'll also cone away with a deeper understanding of the 
evaluaticn process. 

See you cn the (DATE) I 

Sincerely, 



'^ame 

Director 
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WORKSHOP CQNFIKWATIC:i 



Name — 

I WILL attend the Evaiuator Wor)cshop on (DATE) . 

I WILL NOT attend the Evaiuator Wor)csncp on (DATE) . 

Your suggestions will help us address your needs niore fully: 



Please return this woricshop cent irr.ation no Later *^nan {DATE^ . For 
your convenience a stamped pre-aadressed envelope nas oeen enclosed. 

Feei tree call tne Assessment Center at ^08) 555-9390 if you have 
questions . 



UXXJ 

Assessment Center 
1S453 Parx Road 
Hometown, IL 61118 
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WORKSHOP REMIKDEH 

JUST A REMIHDEH 
THE ASSESSMENT CrTTTER r/AL'JATOR WORKSHOP 

will take place cn 
(DATE) 
at (TIME) 
wo will be r.eetinq at 
(PLACE) 

Some of t.'^fe topics ycu ana your feilcw eva-^uaccrs nave suqgescedl for 
the worKsnop include: 



SEE VOU THERE 1 
11 
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WORKSHOP MATERIALS 



Evaiuacor Pacxets 
(see separate list) 

Sample Essays 

Staff Mame Tags 

Department Agendas 
(trore detailed than general Workshop Agenda) 

Chart - Objectives 

Flip Charts, Mar)cers 

Overneads/ Projector 

Extra paper, pens 



12 



EVALUATOR PAOOSTTS 



Name Tags 

Evaiua-^-.or name and area(s) of expemse. 

Woricshop Agenda 

Porrfoiio Handboo)c 

Explains phxic jphy and nature of Porrfoiio. Covers the 
requirements ot an essay, aecnanics of procedures, fees, etc. 

Evaiuator HandboOc 

Explains guidelines and procedures to be followed. Includes 
latitude of evaluators, payment schedule, etc. 

Suggested Subject Areas for Learning Experience Essays 

Most common essay topic titles fcroKen down by departments and 
index/course numbers. 

Topic Outlines 

A selection of outlines prepared as guides for writing specific 
essay topics. 

Update Sheet 

Highlights na-)or policy changes since previous editions of the 
Portfolio and/or Evaiuator Hanaccoics. 

Wor>:shop Evaluation Sheet 

Pertinent questions to determine usefulness of session. 

Ice Sreaxer 

Humorous pre-quiz to warm up nhe participants. 

Group Evaluation Discussion Sheet 

Offers several discussion points fcr use during the Group 
Evaluation exercise. 

Buddy List 

Pairs evaluators for Peer Evaluation exercise. 
BlanK Paper 

For note ta)cing and Peer Evaluation exercise. 
Participant List 
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WORKSHOP ET/ALw'ATICM 

Pleasa help us to plan future offerings by responding to the following 
questions. Feel free to use the reverse of this sheet. 

1. Did the workshop accomplish its stated objectives? 
(circle one) 

YES MO 

2. Did the workshop present infcrr,aticn t!iat riight influence your 
evaluations in tir.e future? 

(circle one} 

YES HO 

3. Has the opportunity to meet with your peers been helpful? 
(circle one) 

YES MO 

4. A. What did you like most about t.':e workshop? 



B. What iid you like the least? 



5. What areas would you like to know aore about? 



General comments: 



BESTC0PV^VA1LABLE 
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WORKSHOP rOLLCW-UP 



(DATE) 



D«ar Evaiuator: 

On benaif ot the enrire Assessmcnr cenrer sxJxtT. tnamc you for your 
participation m tne Evaiuator WoiDcsnoo ].a«t v««x* w« nope you will 
find tnat tne new ideas and practical suqgcationa generated at the 
session will be of assistance to you as an evaiuator at prior learning. 
Enclosed is a copy of t.ne worxshop Proceedings for your reference. 

Again, we appreciate your attenaance and hope you will be anie to lom 
us at the next Evaiuatcr worxsnop. 

Sincerely, 



Name 

Director 
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NATIONAL-LOUIS UNIVERSITY 
EVALUATOR SUPPLEMENT 
TO 

PORTFOLXO HANDBOOK 



OCTOBER 1991 



This docuaant ahouXd s«rv« as a suppXeaant to tha Katlonal-Louia 
Univarsity (NLU) Portfolio UandiaooK &nd Praaantation Handouts 
(Suqgestad Topic Aroas and Topic outlines) • All Asssssasnt csntsr 
?acultY Evaluatora should hava currant copias ot thasa publications 
and uaa them aa guides in their vorJc. 
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EVAIUATOR SUPPLEKErr TO PORTFCIIO HANDBOOK 



PRIOR LEARNING 

Students applying for admission to universities today do not 
necessarily fit the description of the traditional college student. 
Today's college student is older, frequently attending part-tine, 
and nas usually been involved m nany informal learning activities 
since leaving nigti school. 

Colleges and universities are responding to these students by 
offering programs which recognize their specific learning 
characteristics. Increasing niianers of colleges and universities 
are awarding credits for the learning acquired through non-college 
experiences. This learning, frequently called prior experiential 
learning, may occur through enpioynent, volunteer woric, community 
services, travel, military duties and hosecaicing. 

The American Council on Education (ACE) and the Council on 
Postsecondary Accreditation (COPA) have acxnowledged that "American 
society abounds in resources for learning at the postsecondary 
level. Associations, business, government, industry, and unions 
sponsor formal instruction. In addition, ^independent study and 
reading, woric experiences, the nass media, and social interaction 
contribute to learning and competency'* (ACE/COPA Statement on 
Awarding Credit For Extramstituticnal Learning) . 

NLU evaluates prior learning through a portfolio assessment process. 
This process is designed to fulfill the ACE and COPA mandate to 
"assess extrainstitutional learning as parr of the credentialing 
function" of postsecondary institutions. 

Students petition for credit fcr prior learning through 
documentation of NLU-recognized professional licensure or 
certification and the submission of Learning Experience Essays tor 
knowledge acquired via life or vorx experiences* Learning 
Experience Essays are analytical reports wnich describe what 
students have learned (concepts or theories) and how they have 
learned it (experiences and applications). It is not enough lust to 
describe the concepts learned or the experience itself. Students 
must demonstrate an ability to integrate tr.eory with practical 
application. 

The function of the faculty evaluator is t.'^e assessment of the 
Learning Experience Essays. 
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EVftLWTQR PEQUIREMEHTS 



In addition to a general interest in the field of prior learning 
assessaent and academic qualifications, the following are also 
essential: 



* Belief that college level learning can and does occur 
outside the fomal classroom setting 



* Willingness to adapt academic standards and expectations to 
a nontraditional assessment format 



* Flexibility and cpen-ramdedness in assessment issues 



* Commitment to the assessment process 



* PROMPTNESS m ccspleting evaluations 



Adapted from The Faculty Guide to Portfolio Assessment: , office of 
Tearing and Assessment, Thomas A. Edison State College. 
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GENERAL GUTDE;.INES FOR rgSAY ASSESSMENT 

In evaiuaring essays you need t:o assess tixa following: 



understanding of Tlieory (General Concepts) 
Personal Applications (Specific Experiences) 
Integration of Tlxeory aad Applications 



Vritten Presentation 



The appropriate balance between theory (general concepts) and 
application (specific experiences) varies depending upon the topic 
area. For example, health science areas would tend to require a 
greater number of general concepts as compared to applications; 
psychology might require few general concepts but a greater amount 
of personal application. As a rule of thuau, the more tecnnical an 
area is the higher number of concepts will be required; the more 
personal an area is the greater number of personal applications will 
be required. In addition* students should be able to integrate 
their specific experiences with the concepts they are discussing. 

written Presentation 

While the evaluator is primarily evaluating content, ail learning 
experience essays should be written in an appropriate college-level 
manner. The raininun length requirement is five (5) typed pages, but 
we recommend seven to ten (7 - 10) pages as a rule of thumb. Double 
spacing and one inch margins are standard. It is appropriate for 
the student to cite any borrowed worK or ideas* 

Documentation 

Students are required to submit documentation for their learning 
experience essays. The documentation is to provide evidence for the 
experience (s) wnicn the student describes in the learning experience 
essay. For example, a student writing an essay on performance 
review may include a letter or 30b description from an employer 
stating the performance reviews are an expected part of the 
student's position, A listing of the types of documentation can be 
found in the Portfolio Handhoojc . 



If a title chosen by a student does not accurately reflect the essay 
content and learning outcomes, cross it out and write m a more 
appropriate one. Title changes should be briefly explained in the 
comments section. FeeabacK to students snould be as specific as 
possiole, indicating wny credit was awarded or denied. 
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CQtXgGE LEVEL LEARNING 

Although learning gained is often extreaely valuable to students^ it 
is not nacassarily applicable to a college degree. Learning on a 
strictly personal level or in areas noc appropriate to a college 
ctxrriculua should NOT be awarded credit* Likewise, learning 
specific only to the situation in which it is acquired or not 
equivalent in quality to college level wor)c should not receive 
credit. 



AWARDING CREDIT 

Just as in the evaluation of traditional college course work, each 
evaluator will use individual and somewnat subjective standards in 
assessing learning* However, general guidelines outlined here 
should be followed to ensure consistency and academic integrity* 
Essay evaluation standards should be equivalent to those used in 
evaluating classroom learning, but neither more nor less rigorous* 

Letter grades are not assigned, but credit should be awarded to 
students demonstrating competency at the '*C** level or higher* You 
will see a wide range of ability and level of expertise among 
portfolio students; do not equate credit awards ^o traditional 
letter grades. 

Credit requested will be for a total of 3 to 5 QK (see Petition 
Sheet - Appendix A) * Basically studencs are petitioning for credit 
equivalent to what would be awarded in a traditional college course* 
Therefore, do not grant in excess of the total requested. 

Do not hesitate to request revisions or zo deny credit if the essay 
or documentation so deem these actions. Zn some instances you night 
vish to request additional documentation to clarify or add to what 
has been provided* If students seem to xjiow only part of the 
expected learning outcomes, use your discretion to award from 1 to 4 
credits* you are the conrent expert. 

Your options for awarding credit are as follows: 

1. Award requested credit 

A* Essay merits requested credit 

B* Award requested credit, pending receipt of documentation 

by an Assessment Counselor 
2* Award partial credit 

A* Allow for a re%nrite and/or addendum to earn 

additional credit (please stark '^Returned for Revision** 

space on the Petition Sheet) 
B. State that the essay merits only partial credit due to 

its limited nature 
3 . Award no credit 

A* Allow a rewrite (a9ain, please nark the appropriate 

space on the Petition Sheet) 
B. Reject the subject area as non-collegiate 
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When returning an essay, always fill out t!:e comments section of the 
Petition Sheet as fully and clearly as possible using black ink for 
xeroxing purposes. A formative versus sumaative evaluation 
tremendously increases the educational value of the portfolio 
process. All students should he allowed at least one essay revision 
and are limited to two revisions. 

If after the first revision you grant zero or limited credit but do 
not feel that an additional revision would be appropriate, please be 
sure to communicate this in your comments. The student will receive 
a copy of this petition sheet. If you have written any comments in 
the body of the essay or on separate pages, these pages will also be 
copied and sent to the student. 

When returning the evaluated essay, you will also need to sign, 
date, and note the credit award on the attached Transmittal/Payroll 
Sheet (Appendix B) . Keep the yellow copy tor your records, and 
return the original along with the petition sheet, essay and 
documentat ion • 

Requested turnaround time for essay evaluations is two to three 
weeks. Please try to return all materials within that timeframe, or 
let us know if you need more time to complete an evaluation. 



EVALUATQR FEEDBACK 

In order to avoid misunderstandings, evaluators find it helpful to 
use constructive, non-]udgmental terminology in denying credit. 
Most students spend a great deal of time cn their learning 
experience essays and are very appreciative of written feedback. 
You are urged to write comments that will ce of the most help to 
students • 

A positive comment might precede a request for additional 
information, followed by specific suggestions to help the student 
focus on learning outcomes. If follow-up is necessary, be as 
specific as possible as to what the student should do to complete 
the assignment. Comment on the nature and quality of the evidence 
of learning presented and the extent of the student's knowledge. We 
suggest that you note one or more strengths and weaknesses to 
provide more tangible feedback to the student. 



ESSAY REVISTQNS 

When a student revises an essay, the Assessment Center will send the 
revision with the original essay back to the initial evaluator. The 
Petition' and Transmittal/Payroll Sheets (Appendices A and B) will 
indicate a revised essay. Students may generally submit up to two 
revisions. Hote that for record keeping and payroll purposes, all 
original essays and revisions roust be sent through the Center; 
should you receive materials directly from a student, forward them 
to the Assessment Center for completion of the proper paperwork. 
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CONTACT WITH STUDENTS 

occasionally students need further clarification on evaluated 
essays, although an Assessment Counselor vill always attempt to 
handle the situation first. At the student's request, however, we 
might asic you to communicate directly witi that student. Typically 
we will send you a note explaining the request, along with the essay 
in question. 

PlWf MKe an effort to contact the etmdaat bv phone or mmj i 
YQUr earliest: eanveniance: Direct evaiuator-atudent contact often 
results m a stronger rewrite and enhances the evaluation process. 

When your conference has been completed, please return the essay to 
the Assessment Center. A note detailing any pertinent outcomes of 
the conversation helps us in working with the student in the future. 
Generally we ask evaluators to contact students, not vice versa. 
Evaluator phone numbers and addresses are not distributed to 
students. 

WHBH EVXI.DXTINO ZaSKYB, PLEASE COKTXCT THE CENTER XT YOU HAVE AMY 
QUE8TI0MS, PROBLEM8 OR COMMENTS. WE ARE HERE TO HELP YOO, AND WE 
VALUE YOUR INPUT* 

THE ASSPcyc^MENT CgNTCT 

The Assessment Center makes every efforr to advise students of the 
above procedures. Additionally, during tr.e Portfolio Presentation 
students receive a packet containing Suggested Topic Areas 
and Topic Outlines. We recommend that students use available 
outlines only as guides in writing their essays, unless the outlines 
stipulate specific requirements. Therefore, students do have a 
certain amount of latitude m developing essay content. 

Occasionally the Assessment Counselor will note comments to you on 

(Appendix 3). These comments and your 
notations in return are not seen by the student. Typically, the 
counselor might alert you to possible overlap of content or other 
conflicts. In some instances the counselor might also attach 
transcripts or previous essays to assist :n determining duplication. 
Remember, you are the subject natter specialist; we need your 
assistance in making these ]udgments, since duplication of credit ig 
not allowed . '■-^ 
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COMMON ASSESSMENT ERRORS 

The Council for Adult and Experiential Learning (CAEL) has 
identified some common assessment: errors tihat evaiuators should try 
to avoid: 

* Evaluating too liberally or too severely 

* Rating everything on the average r avoiding the extremities 

* Allowing one trait or aspect of performance, either positive 
or negative, to influence the evaluation of other factors 
(halo effect) 

* Judging learning according to a stereotype or strongly held 
personal attitude 

* Prejudging by an initial impression 

* Rating an essay lower than usual if the previous essay was 
outstanding, or higher than usual if the previous essay was 
weaic (contract effect) 

From New college of St. Edward's University Portfolio Assessment 
Guidelines for Assessors. 



REIMBURSEMENT FOR TTTARNTNG EXPfrRIEKCE ESSAYS 

Evaiuators are paid for learning experience essays, whether or not 
credit is assessed. Payment is S20.00 per essay for an initial 
evaluation and SIO.OO per rewritten essay. 

Payroll is calculated from the Transmittal/Payroll Sheets 
(Appendix B) that you return with the essays. Please be sure to 
sign and date these sheets and fill in the amount of credit awarded. 
Any comments directed specifically to the Assessment center should 
be written on this page, since it will not re copied for the 
student. Remember to keep your yellow copy in order to compare with 
your monthly Payroll Summary (Appendix C) and paychccK. 

The Assessment Center Payroll is computed at the end of each month, 
and payment is made at the end of the following month. For example, 
the payment for essays evaluated in September would appear in the 
October paychecK. For full-time NLU faculty, payment is listed as 
••overload*' on your paycheck. Should you have any questions 
concerning reimbursement, please call the Center. 
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Appenaix A p^Qj^ ACADEMIC CREDIT FOR EXPEFJENTIAL LEARNING 
I 1 Original Submission CH Rovisad Submission Group Name ana #: 



Name: 



Course # Dept.. 

"nile: 



Credit Requested: Documentation on Pages: 

Briefly descnbe the learning acquired from your expenence(S) m the space oeiow. 



Stuoent Signature 



To be completed by evaiuatcr 



1 Examoie of Involvement 
2. Learning Outcomes: 

3 Reltea:cn/ Application: 

4 Written Presentation: 

Evaiuaior Comments: (Use reverse side if needed) 



Weak Acceotable Superior 

1 2 3 

1 2 3 

1 2 3 

t 2 3 



Credit Peccmmenoed: [Z] Returned tor Revision 

9y Date: 

Page 10 
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Appendix 3 

BSSI — I ^ ^' ^^^^ ^^ "^ LOMBA RD. ILLINOIS 60148 - 708»691»9390 



TRANSMiTTAUPAYROLL SHEET 



EVALUATOR 



TOPIC AREA 



STUDENrS NAME 



GROUP 



ORIGINAL 



1st REWRITE 



LEARNING EXPERIENCE TITLES 



I 



I 1 



2. 



COMMENTS FROM ASSESSMENT CENTER 



2nd REWRITE 



CREDIT AWARD 



SIGNATURE OF COUNSELOR 



DATE 



I COMMENTS TO ASSESSMENT CENTER 



SIGNATURE OF EVALUATOR 



DATE 



Page 11 

WHITE . ORIGINAL COPY • CANARY » EVALUATOR CCPV • PINK • PILE COPY 



ERLC 



193 



182 



Appendix C 



EVALUATOR: 



E. Val Uator 



Student Name 



ASSESSMENT CENTER PAYROLL SUMMARY 
MONTH: September, 1991 



Essay Title 



13 t Crealt 2nd Credit 

Heading Award Reading Award 



Doe, J, 
Smi th . S « 
David, D. 



Nutrition x 
Pregnancy/childbircn x 
Infertility x 



0 
2 
0 



ST. LOUIS 
Wright, K. 
James, T. 

MCLEAN 
Williams. D. 
Thomas. M. 

ATLANTA 
Louis . R . 
Jones, J. 

TAMPA 
Hunter. 



Nu t r 1 nion 

Heai t h/ nut r 1 1 ion 



Nutrition 

Pregnancy/c.^. ildbircn 



Nutri t ion 

Pregnancy/childbircn 



Nutrition 



5 
0 



0 
5 
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rVXLUXTOR WORMKOP 

Loruoard Campus 
Ocrober 19, 1991 



9:00 - 9:30 Coffee and Roils 

Distritution of worKstiop Materials 

9:30 - 9:45 Welcome 

WorKshop Objectives 

Explanation of workshop Materials 

9:45 - 10:15 The Evaluation Process 

Role of Assessment Center Personnel 
Role of ^valuator 

10:15 - 11:00 Evaluator Panel Discussion 

What IS a Learnmq Experience Essay? 
Scott HecK, Moderator 
Randee Lawrence 
Craiq Mealman 

11:00 - 11:10 BreaK 

11:15 - 12:15 p.m. Group Evaluations / Discussion 

(see group assignments in Materials) 

12:15 - 12:45 Closing Comments 

worKsncp Evaluations 

1:00 Luncft 

Special Presentation 
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